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The  purpose  of  this  study  was  to  develop  information  showing  the 
relationship  of  one  outcome  measure,  the  general  written  qualifying 
examination,  to  measures  of  selection  and  training  of  doctoral  stu- 
dents in  the  University  of  Florida  College  of  Education.     The  quali- 
fying examination  was  comprised  of  five  academic  areas.     The  areas 
were:     educational  administration,  psychological  foundations  of  edu- 
cation and  personnel  services,  sociological  foundations  of  education, 
curriculum  and  instruction,  and  philosophical  and  historical  founda- 
tions  of  education. 

The  procedures  of  this  study  included  developing  a  set  of  five 
linear  equations  to  yield  the  combination  of  selection  and  training 
measures  most  predictive  of  success  in  each  of  the  five  academic  areas 
tested  on  the  qualifying  examination.     A  hierarchical  clustering  tech- 
nique was  then  used  to  group  the  five  equations  in  terms  of  their  homo 
geneity  to  determine  the  extent  and  pattern  of  predictive  agreement 
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between  the  five  academic  areas.    The  clustering  procedure  ultimately 
produced  a  single  equation  from  the  original  five  to  achieve  predic- 
tion of  composite  success  with  maximum  predictive  efficiency.  The 
predictive  qualities  of  three  subsets  of  variables,  pre-admission 
variables,  post-admission  variables,  and  program  variables ,  were 
also  investigated. 

The  population  for  this  study  was  all  357  doctoral  students  who 
took  the  general  qualifying  examination  in  the  College  of  Education 
during  the  years  1963  through  1969.     A  total  of  78  independent  vari-  ' 
ables  were  investigated.     These  variables  were  of  two  forms:     45  were 
obtained  directly  from  the  student  files,  and  33  were  formulated  by 
combinations  of  directly  obtained  variables.     The  criteria  were  scores 
received  in  each  of  the  five  academic  areas  of  the  qualifying  exam- 
ination.   Since  students  were  scored  by  two  graders  in  each  academic 
area  the  effective  number  of  cases  was  714. 

The  equations  developed  for  prediction  of  each  area  of  the 
examination  ranged  from  19  percent  to  29  percent  in  maximum  predic- 
tive ability.     The  combination  and  weighting  of  predictor  variables 
for  each  of  the  five  equations  was  rather  unique  upon  inspection. 
This  uniqueness  was  supported  by  a  difficulty  in  grouping  the  equa- 
tions into  common  predictive  clusters.    An  equation  to  predict  the  aver 
age  of  all  areas  on  the  examination  had  a  maximum  predictive  ability 
of  36  percent.     The  predictive  qualities  of  each  of  the  three  subsets 
of  variables  referred  to  above  were  highly  significant  in  overall 
prediction. 
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Several  conclusions  were  drawn  from  this  study.    First,  looking 
at  just  the  variables,  the  importance  of  grades  in  doctoral  study  as 
a  predictor  variable  in  all  areas  was  noted.    This  importance  suggestec 
that  grades  could  be  a  very  reliable  means  of  detecting  difficulty  in 
doctoral  study.     Two  other  variables  were  somewhat  common  in  adding 
to  multiple  prediction.    These  variables  were  score  on  the  Verbal 
Aptitude  Test  of  the  Graduate  Record  Examination  (GRE)  and  the  square 
of  an  applicant's  age.    Several  variables  were  also  cited  because  of 
their  lack  of  predictive  ability.     One  such  variable  was  the  score  on 
the  Quantitative  Aptitude  Test  of  the  GRE.    Either  the  quantitative 
score  of  the  GRE  was  of  little  value  in  predicting  desired  outcomes  in 
these  doctoral  students,  or  the  skills  predicted  by  the  quantitative 
score  of  the  GRE,  though  desirable,  were  not  tested  on  the  qualifying 
examination. 

Second,  looking  at  the  equations,  the  uniqueness  of  each  area 
equation  suggested  that  a  par ticularlized  admission  procedure  depend- 
ing on  prospective  major  would  be  profitable.     Since  the  average  of 
all  areas  was  more  predictable  than  any  single  area,  it  is  possible 
that  considerable  variation  existed  among  the  graders  in  each  area. 
It  was  mentioned  that  such  possible  variation  could  be  reduced  by  the 
adoption  of  standardized  testing  procedures  on  the  qualifying  examin- 
ation. , 
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CHAPTER  I 
THE  PROBLEM  ; ,  ; 

\ 

Professional  educators  are  increasingly  aware  of  the  importance 
of  having  a  system  of  accountability  at  all  levels  of  education.  This 
awareness  has  been  noted  by  several  sources  (Barro,  1970;  Florida 
Assessment  Plan,  1971;  Lessinger,  1970;  Lieberman,  1970).  Accountabil- 
ity has  frequently  been  presented  in  the  literature  as  a  management 
approach  to  education  characterized  by  three  primary  operational  phases 
input,  process,  and  output  (Barro,  1970;  Durstine,  1970;  Manning,  1970) 
The  essence  of  accountability  is  to  provide  a  basis  for  management 
procedures  in  education  which  will  ultimately  reconcile  the  input  of 
personnel,  money,  and  material  with  the  output  of  desired  student  de- 
velopment.   The  goal  of  establishing  accountability  is  to  maximize 
the  efficiency  of  an  educational  program  by  providing  policy  makers 
with  reliable  feedback  information  on  which  to  base  decisions.  ' *~ 

Durstine  (1970),  Lieberman  (1970),  and  Manning  (1970)  have  sug- 
gested that,  in  order  to  provide  the  kind  of  information  which  will 
permit  more  effective  decision  making,  an  informational  scheme  or 
framework  is  necessary.     That  is,  some  comprehensive  structure  for 
integrating  information  from  measures  of  input,  process,  and  output 
is  requisite  to  assessing  vAiich  choices  within  the  limits  of  available 
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resources  are  most  likely  to  result  in  desired  outcomes.     The  very 
basis  for  responsible  decision  making  is  adequate  determination  of  pos- 
sible outcome  effects  among  the  choices  for  the  decision.     For  example, 
a  responsible  decision  in  a  matter  of  policy  requires  reasonable  a  pri- 
ori evidence  that  the  policy  being  considered  for  adoption  will  most 
effectively  and  efficiently  bring  wanted  results. 

Durstine  (1970)  also  pointed  out  that  the  problem  of  organizing 
information  into  relational  patterns  is  of  prime  importance  in  the 
whole  question  of  accountability.     He  recognized  that  valid  measures 
of  accountability  are  critical  to  successful  management,  but  he  indicated 
that  their  intended  purposes  can  be  served  only  if  they  are  synthesized 
".  .  .  into  an  informational  base  for  decision  making"  (p.  236).  Though 
Durstine  was  talking  about  an  accountability  information  system  in  a 
general  sense,  there  is  specific  concern  for  creating  better  developed 
systems  of  information  for  management  purposes  in  graduate  education. 
This  concern  has  been  reflected  in  several  recent  reports  of  national 
scope  (Burns,  1970;  Carnegie  Commission,  1968;  Manning,  1970;  U.  S. 

1 

Dept.  of  HEW,  1969).     Additionally,  the  report  by  Burns  (1970)  sug- 
gested that  graduate  education  at  the  doctoral  level  encompasses  the 
same  difficulties  as  graduate  education  in  general  except  that  the 
difficulties  are  likely  to  be  more  acute  at  the  doctoral  level. 

The  national  need  to  establish  an  information  system  for  grad- 
uate education  creates  a  task  for  graduate  institutions  since  they  are 
an  obvious  source  of  such  information.     In  the  first  place,  it  is  im- 
portant that  graduate  institutions  collect,  maintain,  collate,  and 
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disseminate  basic  information  related  to  their  operation  as  input  to 
more  global  information  systems.     Secondly,  graduate  institutions  should 
develop  information  systems  for  their  own  policy  making  in  order  to 
attain  maximum  return  on  allocated  resources.  , 

The  first  task  is  primarily  one  of  cataloging.     This  does  not 
imply,  however,  that  it  is  a  simple  clerical  procedure.    Developing  a 
systematic  file  system  requires  very  careful  design  and  evaluation 
through  time  to  determine  which  information  to  file  and  in  what  form 
and  order.     The  second  problem  requires  some  means  of  integrating 
cataloged  information  to  reveal  relational  patterns  between  accounta- 
bility measures. 

As  shown  by  Barro  (1970),  integrating  accountability  measures  to 
show  patterns  of  relationships  places  demanding  requirements  on  the 
integrating  procedures.     The  procedures  should  permit:     (a)  a  large 
number  of  cases  to  be  processed,   (b)  multiple  factors  to  be  considered, 
(c)  interactions  between  factors  to  be  considered,  and  (d)  empirical 
comparisons  of  strength  of  relationships  to  be  made.     A  set  of  basic 
procedures  which  meets  these  requirements  has  been  described  by  Ward 
(1962)  and  Bottenberg  &  Ward  (1963)  in  the  form  of  a  series  of  appli- 
cations for  fixed-effects  linear  regression  techniques.    More  specific 
applications  of  the  Bottenberg  &  Ward  techniques  reported  by  Botten- 
berg &  Chris tal  (1968)  and  by  Chris tal  (1968a,  1968b)  offer  extremely 
powerful  tools  for  integrating  accountability  measures.  Bottenberg, 
Ward,  and  Chris tal  developed  their  procedures  while  on  contract  to  the 
U.  S.  Air  Force  primarily  to  satisfy  management  requirements  for  in- 
formation on  which  to  base  personnel  decisions.     Their  techniques  in 
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general  have  been  used  extensively  and  successfully  by  the  Air  Force 
in  accountability  situations  involving  utilization  of  personnel.  An 
opportunity  exists  to  apply  these  techniques  to  alleviate  a  difficulty 
in  the  University  of  Florida  College  of  Education. 

The  College  of  Education  has  no  integrated  information  relating 
any  outcome  measure  to  measures  of  selection  and  training  in  doctoral 
study.     Fairly  extensive  student  records  are  available  in  the  college 
which  provide  an  information  source  for  accountability  measures,  but 
the  information  has  never  been  systematically  collated  and  integrated 
into  relational  patterns.    One  outcome  measure,  the  general  written 
qualifying  examination,  was  administered  to  all  doctoral  students  in 
a  standard  way  through  the  period  1963-1969.     During  this  time,  357 
students  took  the  examination. 

Writing  about  the  examination  in  an  earlier  time,  Molbert  (1960) 
reported  that  this  examination  was  both  a  measure  of  achievement  and 
a  learning  device.     Though  there  was  a  variety  of  opinions  among 
surveyed  faculty  and  students  about  the  meaning  of  the  examination, 
the  majority  of  both  perceived  the  examination  as  a  measure  of  achieve- 
ment.   Molbert  saw  as  a  problem  a  lack  of  information  about  the  rela- 
tionship of  the  examination  to  other  program  variables.     Her  study 
recommended  that  further  investigations  be  conducted  to  alleviate  this 
problem.     Since  the  examination  was  virtually  unchanged  from  the  time 
of  Molbert 's  study  to  the  time  of  interest  in  this  study,  her  findings 
and  recommendations  are  still  relevant. 

In  another  older  but  still  pertinent  study,  McBride  (1951) 
showed  that  a  lack  of  information  about  the  relationship  between 


outcome  expectations  and  selection  and  training  procedures  in  doctoral 
programs  was  a  traditional  and  universal  problem.    Manning  (1970)  indi- 
cated that  the  lack  of  this  type  of  information  is  still  a  major  problem 
in  graduate  study. 

The  Problem  Stated 
There  is  currently  no  organized  information  available  in  the 
University  of  Florida  College  of  Education  relating  any  outcome  meas- 
ure to  measures  of  selection  and  training  in  doctoral  study.     This  is 
not  only  a  local  problem  but  part  of  a  larger  national  problem.  In- 
formation of  this  type  is  the  basis  of  accountability  information  sys- 
tems and  is  essential  to  effective  management  decision  making.  The 
purpose  of  the  present  study  is  to  provide  information  showing  the 
relationship  of  one  outcome  measure,  the  general  written  qualifying 
examination,  to  measures  of  selection  and  training  of  doctoral  stu- 
dents.    Specifically,  this  study  developed  a  set  of  five  linear 
equations  showing  the  combination  of  selection  and  training  measures 
most  predictive  of  success  in  each  of  five  academic  areas  tested  on 
the  general  qualifying  examination.     The  measure  of  success  was  the 
individual  grade  assigned  by  each  faculty  member  grading  a  question 
in  a  particular  area.    A  hierarchical  clustering  procedure  was  then 
used  to  cluster  the  five  equations  in  terms  of  their  homogeneity  to 
determine  the  extent  and  pattern  of  predictive  agreement  between  the 
five  academic  areas.     The  clustering  procedure  also  produced  a  single 
equation  from  the  original  five  to  synthesize  the  prediction  of  com- 
posite success  with  maximum  predictive  efficiency.     Finally,  the 


predictive  qualities  of  three  groups  of  variables,  pre-admission  var- 
iables, post -admission  variables,  and  program  variables,  were  inves- 
tigated and  reported. 

Significance 

The  problem  presented  immediately  above  has  two  aspects.  One 
aspect  is  the  product,  an  information  base  from  which  management 
decisions  can  be  made.     The  other  aspect  is  the  process,  the  tech- 
niques  or  methods  used  to  produce  the  information  base. 

The  Information  Base 

As  previously  mentioned,  several  recent  reports  of  long-range 
national  scope  have  indicated  the  need  and  intention  of  gathering 
information  about  graduate  students,  graduate  studies,  and  graduate 
institution  strategies  in  order  to  evaluate  and  make  recommendations 
about  graduate  education  policies  (Burns,  1970;  Carnegie  Commission, 
1968;  Manning,  1970;  U.  S.  Dept.  of  HEW,  1969).     The  primary  concern 
expressed  in  these  reports  was  that  a  basis  be  established  for  re- 
lating means  to  ends  in  graduate  education.     This  concern  was  justi- 
fied on  the  basis  of  the  rapidly  increasing  number  of  applicants  to 
graduate  education  and  the  increasing  national  urgency  to  allocate 
shrinking  federal  and  foundation  funds  to  graduate  education  in  ways 
most  responsive  to  society's  well-being. 

Information  generated  by  the  present  study  can  complement 
national  studies  in  two  ways.     First,  it  constitutes  a  fairly  com- 
prehensive package  of  processed  information  about  a  population  of 
doctoral  students.     Such  information  could  be  useful  as  input  data  ' 
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to  a  national  system.    Second,  planners  at  the  national  level  will 
have  to  make  decisions  about  what  types  and  forms  of  data  to  include 
in  a  national  system.    This  study  has  provided  a  partial  basis  for 
indicating  what  types  of  data  could  be  usefully  included  in  such  a 
system. 

At  the  local  level  this  study  can  be  directly  useful  as  a  base 
of  information  for  policy  decisions.    As  was  indicated,  no  information 
exists  in  the  College  of  Education  which  shows  the  relationship  be- 
tween  an  outcome  measure  and  measures  of  selection  and  training  of 
doctoral  students.    The  need  for  such  information  has  been  pointed 
out  by  McBride  (1951)  as  a  general  problem  in  doctoral  programs  and 
Molbert  (1960)  as  a  specific  problem  in  the  doctoral  program  in  the 
University  of  Florida  College  of  Education. 

Integrated  information  relating  measures  of  output  to  measures 
of  input  is  fundamental  to  an  accountability  system.    Though  the  re- 
sults of  this  study  only  partially  contribute  to  the  overall  require- 
ment for  accountability  information  at  the  local  level,  the  informa- 
tion appears  adequate  for  some  policy  considerations. 

As  stated  before,  integrating  accountability  measures  to  show 
relational  patterns  places  demanding  requirements  on  the  integrat- 
ing procedures.     Barro  (1970)  made  this  point  clear  when  he  said 
success  in  accountability  may  depend  ".   .   .  on  the  adequacy  of  the 
statistical  model  in  mirroring  the  underlying  (and  unknown)  input- 
output  relationships  in  education"  (p.  207).     Techniques  developed 
by  Ward  (1962),  Bottenberg  &  Ward  (1963),  Bottenberg  &  Chris tal 
(1968),  and  Christal  (1968a,  1968b)  were  used  in  this  study.  Their 


techniques  were  developed  for  personnel  management  applications  in  the 
U.  S.  Air  Force  and  have  been  used  extensively  in  accountability  situ- 
ations.    To  the  extent  that  these  techniques  were  sufficient  for  the 
needs  of  this  study,  they  may  also  be  seen  as  potentially  useful  for 
integrating  accountability  information  in  a  variety  of  education 
adaptations .  - 

Organization  of  the  Remainder  of  Dissertation 
In  Chapter  I  the  problem  and  its  signficance  have  been  described. 
Chapter  II  provides  a  review  of  related  literature.    The  review  in- 
cludes:    a  brief  section  on  accountability  to  give  perspective  to  the 
study,  a  section  on  literature  dealing  with  prediction  of  ratings, 
and  a  section  on  literature  dealing  with  the  hierarchical  clustering 
method.     The  procedures  used  in  this  study  including  a  description  of 
the  sources  of  data  appear  in  Chapter  III.    Chapter  IV  contains  the  • 
study  of  results,  and  Chapter  V  contains  the  discussion  of  results. 


CHAPTER  II 
REVIEW  OF  THE  LITERATURE 

As  indicated  in  the  previous  section,  the  literature  is  reviewed 
in  three  areas.    First,  a  brief  review  of  literature  on  accountability 
is  provided  in  order  to  give  perspective  to  the  study.    Next,  litera- 
ture dealing  with  prediction  is  reviewed.    Last,  a  review  of  litera- 
ture  on  the  hierarchical  clustering  procedure  is  presented.    A  summary 
of  the  reviewed  areas  concludes  this  chapter. 

Accountability 

The  notion  of  accountability  in  education  is  not  entirely  new, 
but  recent  emphases  and  new  terminology  have  brought  accountability 
into  focus  in  professional  education.    The  sources  reviewed  here  are 
intended  to  give  a  brief  orientation  into  accountability  and  a  per- 
spective to  this  study. 

In  a  summary  review  of  accountability,  Lieberman  (197o)  explained 
there  are  several  forms  of  accountability  in  education,  such  as: 
merit  pay,  voucher  systems,  and  performance  contracting.     He  warned 
against  judging  the  importance  of  accountability  on  the  basis  of  any 
one  of  its  manifested  forms.    According  to  Lieberman,  there  is  a  gen- 
erally common  meaning  associated  with  the  concept  of  accountability. 
He  reported  that,  from  a  common  sense  point  of  view,  "...  there  is 
accountability  when  resources  and  efforts  are  related  to  results  in 
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ways  that  are  useful  for  policy  making,  resource  allocation,  or  com- 
pensation" (p.  194). 

In  addition  to  having  indicated  a  growing  national  and  even  in- 
ternational interest  in  accountability  among  educators,  the  author 
pointed  out  that  accountability  has  attracted  the  attention  of  gover- 
nors and  legislators  across  the  country.     He  reported  that  basically 
there  are  only  two  approaches  to  accountability.    One  approach  is 
through  relating  input  to  output  in  some  meaningful  way  within  an  edu- 
cational program.    The  second  approach  is  through  consumer  choice. 
Lieberman  stated  that  both  these  approaches  require  an  adequate  in- 
formation system  in  order  to  be  effective.     Since  the  present  study 
generated  accountability  information  suitable  for  policy  considera- 
tions, it  conforms  to  Lieberman' s  common -sense  definition  of  accounta- 
bility and  can  contribute  to  the  requirement  he  mentioned  for  effec- 
tive accountability  information. 

Typical  of  efforts  to  establish  accountability  is  the  report  in 
Florida  Assessment  Plan  (1971).     This  report  is  a  state-level  plan  to 
develop  an  accountability  system  in  the  public  schools.     For  purposes 
of  accountability  it  conceptualized  education  in  terms  of  the  "in- 
dustrial model,"  that  is,  as  comprised  of  input,  process,  and  product. 
Principally,  the  report  expressed  concern  that  precise,  measurable, 
and  relevant  outcome  objectives  be  devised  so  that  the  product  can  be 
assessed.     The  position  taken  in  the  report  was  that  process  cannot 
be  measured  directly,  but  its  appropriateness  can  be  inferred  from  the 
product.    As  a  first  step  toward  establishing  accountability,  the 
Florida  Assessment  Plan  intended  to  determine  outcome  objectives  for 
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basic  learning  skills  (such  as  reading)  and  for  vocational  skills. 
The  difficulty  of  devising  outcome  objectives  for  more  advanced  and 
more  general  learning  situations  was  viewed  as  a  problem  in  the  re- 
port.   It  was  suggested  in  the  Florida  Assessment  Plan  that  future 
steps  toward  implementing  accountability  would  follow  after  consolidat- 
ing efforts  in  the  first  step. 

The  main  point  of  interest  from  the  Florida  Assessment  Plan  to 
the  present  study  was  the  recognized  problem  of  devising  outcome  ob- 
jectives for  learning  situations  other  than  skill  acquisition.  Since 
the  present  study  was  concerned  with  accountability  information  for  a 
doctoral  study  program,  it  was  assumed  that  the  outcome  measure  scores 
on  the  general  qualifying  examination  were  valid  within  the  limits  of 
the  "state  of  the  art." 

In  a  proposed  methodological  approach  highly  related  to  the 
present  study,  Barro  (1970)  agreed  with  Lieberman  that  there  is  a 
generally  accepted  meaning  for  accountability  in  education.  Barro, 
however,  stressed  the  fixing  of  responsibility  as  the  main  theme  of 
accountability  rather  than  Lieberman' s  theme  of  justifying  means  with 
ends.     The  rapidly  growing  interest  in  and  use  of  accountability  in 
education  is  seen  by  Barro  as  testimony  to  its  usefulness  rather  than 
its  novelty.     He  detailed  several  ways  that  schools  can  be  made 
accountable.     In  addition  to  those  approaches  to  accountability  men- 
tioned by  Lieberman,  Barro  spoke  of  external  evaluation  programs  and 
community  control.     Like  Lieberman,  Barro  indicated  that  all  approaches 
to  accountability  depend  on  adequate  information  systems  to  be  effec- 
tive.    He  stated  that  two  types  of  information  are  necessary. 


performance  measurements  and  estimated  contributions  to  performance 
measurements  by  various  individuals  and  components  of  the  education 
system.     Although  he  recommended  that  more  and  better  measures  of 
performance  be  employed,  Barro  was  more  concerned  that  the  performance 
measures  be  related  to  specific  teachers,  schools,  and  districts.  He 
seemed  somewhat  willing  to  accept  presently  available  measures  of 
performance  in  order  to  proceed  with  the  more  important  problem  of 
relating  the  performance  to  the  antecedent  conditions.     The  statis- 
tical procedure  proposed  by  Barro  for  relating  performance  to  condi- 
tions was  regression  analysis.     He  also  called  for  the  use  of 
categorical  "dummy"  variables  as  well  as  continuous  variables  in  build 
Ing  appropriate  models.     He  envisioned  the  following  shortcomings  to 
his  statistical  approach:     (a)  intercorrelation  between  supposedly 
independent  variables,   (b)  omitted  variables,   (c)  structural  limi- 
tations of  the  statistical  models,  and  (d)  validity. 

There  was  a  strong  relationship  between  Barro's  proposed  method- 
ology   and  the  methods  used  in  this  study.     Barro's  approach  was  to 
determine  the  amount  of  variation  in  performance  attributable  to  the 
teacher,  the  school,  the  district,  etc.,  by  a  regression  analysis 
procedure.     The  present  study  concerned  more  an  investigation  of 
systemic  variables  with  a  view  toward  procedures  rather  than  individ- 
uals as  the  sources  of  variation  in  performance.     The  shortcomings 
in  statistical  procedures  envisioned  by  Barro  in  his  proposals  were 
also  difficulties  for  this  study.     However,  there  were  some  advantages 
in  using  an  entire  population,  a  fixed-effects  regression  model,  and 
the  hierarchical  dusting  procedure  which,  it  was  believed. 


minimized  the  shortcomings.    Using  an  entire  population  reduced  the 
differential  effects  of  intercorrelation  between  supposedly  independ- 
ent  variables  since  all  cases  were  affected  in  the  same  way.  Since 
the  fixed-effects  regression  model  permitted  investigation  of  varia- 
tion due  to  interaction  between  variables,  one  aspect  of  structural 
limitations  foreseen  by  Barro  was  eliminated  in  this  study.  Another 
structural  limitation,  treatment  of  only  one  dimension  of  output  at  a 
time,  was  reduced  by  use  of  the  hierarchical  clustering  procedure. 

Lopez  (19  70),  like  Barro,  stressed  holding  the  individual 
(teacher,  administrator,  etc.)  accountable,  but,  unlike  Barro,  he 
distinguished  this  from  the  question  of  fixing  responsibility.  He 
maintained  that  responsibility  is  an  element  of  authority  and  cannot 
be  delegated.    Any  lack  of  success  in  programs  of  accountability  to 
date  was  seen  by  Lopez  as  an  indictment  of  the  way  the  programs  were 
handled  rather  than  as  an  indictment  of  accountability  per  se.  He 
proposed  that  accountability  be  established  by  interventions  or 
changes  in  organizational  goals,  attitudes  of  supervisors,  and  tech- 
niques for  evaluating  individual  members  of  the  school  system.  The 
basic  assumption  that  everyone  who  joins  an  organization  does  so  to 
further  its  goals  is  the  key  to  understanding  Lopez's  proposed  system. 
He  suggested  that  more  effective  teaching  can  be  achieved  primarily  by 
better  measures  of  teacher  performance  and  better  communication  be- 
tween teacher  and  supervisor.     His  approach  to  accountability  was 
somewhat  unique  in  that  it  relied  less  on  a  sophisticated  system  of 
information  and  more  on  trust  and  communication  between  individuals 
for  its  effectiveness. 
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It  is  difficult  to  comment  on  Lopez's  proposals  in  terms  of  the 
present  study.    One  fact  must  be  borne  in  mind,  however,'  Lopez  was 
talking  about  accountability  in  public  schools.    In  two  reports  deal- 
ing with  accountability  in  higher  education,  development  of  comprehen- 
sive information  systems  to  achieve  accountability  seemed  to  be  favored 
(Carnegie  Commission,  1968;  U.  S.  Dept.  of  HEW,  1969).    The  quality 
rankings  of  graduate  programs  and  faculties  reported  by  Cartter  (1966) 
indicated  that  graduate  education  will  probably  be  more  sensitive  to 
an  accountability  system  based  on  comprehensive  information.     But,  as 
Lopez  contended,  trust  and  communication  between  individuals  will  un- 
doubtedly play  a  part  in  any  effective  accountability  system,  even  in 
graduate  education. 

The  question  of  accountability  in  graduate  education  was  shown 
further  in  the  position  of  the  Graduate  Record  Examination  (GRE)  Board 
as  indicated  in  two  reports  (Burns,  1970;  Manning,  1970).    Both  of 
these  reports  reflected  the  GRE  Board's  concern  for  greater  accounta- 
bility in  graduate  education.    Manning  reported  the  research  strategy 
projected  for  the  1970s    by  the  GRE  Board.    According  to  Manning,  ten 
research  areas  have  been  outlined  within  the  following  system  elements: 
input  energy  (talent),  distribution  (admissions  process),  modifying 
functions  (educational  process),  and  output  measures   (graduate  school 
or  postgraduate  achievement).     The  purpose  for  proposing  to  conduct 
this  research  was  reported  as  a  series  of  goals  for  greater  accounta- 
bility.    Burns'  report  gave  the  results  of  a  45-item  questionnaire 
sent  to  all  287  member  graduate  schools  of  the  Council  of  Graduate 
Schools.     The  survey  dealt  with  admissions  and  fellowship  granting 
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policies  and  was  reported  in  categories  based  on  the  number  of  doctoral 
degrees  awarded  in  1967-68.     Results  of  the  survey  revealed  a  variety 
of  facts  about  selection  and  fellowship  awarding  policies.     One  of  the 
most  interesting  facts  in  terms  of  the  present  study  was  the  tendency 
of  graduate  schools  to  conduct  few  follow-up  studies  on  the  success 
of  their  policies. 

Of  the  two  GRE  Board  reports,  the  report  by  Manning  was  perhaps 
the  more  pertinent  to  this  study  because  it  dealt  with  the  problem  of 
building  an  information  system  for  decision  making  in  graduate  educa- 
tion. Another  publication  dealing  with  development  of  an  information 
system  for  accountability  in  education  is  given  below. 

Durstine  (197  0)  was  concerned  with  developing  a  total  accounta- 
bility information  system  suitable  for  all  levels  of  education.  The 
system  proposed  by  Durstine  was  not  anticipated  by  him  to  be  massive  or 
complex.    He  did,  however,  make  the  point  that,  in  order  to  be  effec- 
tive, an  accountability  information  system  must  permit  the  synthesis 
of  accountability  measures  into  relational  patterns  suitable  for  de- 
cision making.     His  contention  supported  the  worth  of  conducting  the 
present  study. 

Prediction  of  Ratings 

The  outcome  measures  in  this  study,  grades  on  the  general  written 
qualifying  examination,  were  faculty  ratings  of  student  performance. 
Since  the  present  study  predicted  these  ratings  from  a  combination  of 
factors,  literature  dealing  with  prediction  of  ratings  is  reviewed  here. 

Early  studies  of  prediction  were  correlational  studies  between  the 
variables  of  interest  and  the  predictor  variables.     In  one  such 
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investigation,  Platz  (1959)  studied  the  relationship  of  five  predictor 
variables  and  four  criterion  variables  of  success  among  124  doctoral 
students.    The  predictor  variables  considered  were  Miller  Analogies  Test 
(MAT)  score,  score  on  an  objective  comprehensive  test  administered  at 
admission,  overall  undergraduate  grade  point  average,  undergraduate 
average  in  science  and  math,  and  undergraduate  average  in  psychology- 
courses.    The  criterion  variables  were  graduate  honor  point  average, 
grade  on  the  doctoral  preliminary  exam,  faculty  rating  of  potential 
professional  contribution,  and  faculty  rating  of  potential  scientific 
contribution. 

Of  primary  interest  for  purposes  of  the  present  study  were  cor- 
relates Platz  found  to  the  doctoral  preliminary  exam.     The  single 
highest  correlate  was  the  undergraduate  average  in  science  and  math. 
It  correlated  .27  with  the  doctoral  exam.     The  highest  two-factor 
multiple  correlation  was  .36  using  MAT  scores  and  undergraduate  average 
in  science  and  math. 

In  another  correlational  study,  Robertson  &  Nielsen  (1961)  found 
the  literature  to  be  very  disappointing  in  producing  predictors  of  suc- 
cess in  graduate  school.     Correlations  between  MAT  scores  and  faculty 
ratings  or  graduate  grade  point  averages  varied  from  .16  to  .68.  They 
reported  that  the  psychology  department  at  the  University  of  Florida 
was  beginning  to  show  increasing  confidence  in  the  aptitude  tests  of 
the  Graduate  Record  Examination  (GRE)  as  a  predictor  of  success. 
Robertson  &  Nielsen  selected  50  doctoral  students  who  entered  the  de- 
partment between  1955  and  1959  and  had  9  of  the  11  faculty  rank  them 
on  a  five-point  scale  indicating  the  degree  to  which  the  student  possessed 
the  intellectual  capacity  to  complete  the  Ph.  D.     Only  ratings  where 


the  faculty  member  claimed  to  have  observed  some  prior  tangible  evi- 
dence of  ability  in  the  student  were  used.     These  researchers  found 
a  correlation  of  .29  between  the  average  GRE  aptitude  score  and 
faculty  rating.     This  correlation  was  higher  than  that  of  either  of 
the  two  aptitude  scores   (verbal  or  quantitative)  individually,  and 
it  was  significant.     Undergraduate  grade  point  average  for  the  last 
two  years  correlated  at  only  .16  and  was  not  significant.    The  multiple 
correlation  of  the  average  GRE  score  and  the  undergraduate  average 
for  the  last  two  years  was  .44  and  significant.     The  predictive  possi- 
bilities of  both  first- and  last  two-year  undergraduate  averages  were 
investigated  in  the  present  study.  ■ 

Hoyt  (1966)  looked  beyond  college  and  reviewed  the  literature 
discussing  the  correlation  between  college  grades  and  adult  achievement 
He  reported  studies  conducted  in  a  number  of  fields  to  ascertain  the 
relationship  between  college  grades  and  later  success.     Hoyt  concluded 
".  .  .  that  college  grades  have  no  more  than  a  very  modest  correlation 
with  adult  success,  no  matter  how  defined"  (p.  75).     The  predictive 
efficiency  of  a  variety  of  both  intellective  and  non-intellective 
factors  was     considered  in  the  present  study.  , 

The  possibility      of  predicting  ratings  or  judgments  by  regres- 
sion models  is  a  more  recent  technique.     In  most  cases,  linear  models 
are  developed  because  of  their  comparative  ease  of  computation  and 
accuracy  of  prediction.     Hoffman  (1960)  in  a  classic  investigation 
into  the  possibilities  of  approximating  psychological  clinicians'  judg- 
ments with  regression  equations  found  that  this  could  be  done  quite 
satisfactorily.     He  also  investigated  the  relative  merits  of  a 
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figurational  model  compared  to  a  linear  model  for  this  purpose.  His 
finding  was  that  the  linear  model  was  sufficiently  accurate  to  the 
point  that  the  additional  difficulty  in  using  the  conf igurational  model 
did  not  justify  the  small  increase  in  accuracy. 

Schenck  &  Naylor  (1968)  warned  of  a  shortcoming  in  using  regres- 
sion analysis  to  capture  the  strategies  of  people.    The  caution  stemmed 
from  the  tendency  for  linear  consistency  to  increase  as  the  intercor- 
relation  between  pairs  of  variables  increases.     Since  an  entire  popula- 
tion was  used  in  the  present  study,  no  differential  results  should  have 
occurred  and  the  effect  described  by  Schenck  and  Naylor  should  have 
been  negligible  to  the  relative  findings. 

In  a  very  recent  article,  Dawes  (1971)  explored  the  possibility 
of  predicting  an  admissions  committee's  judgments  in  the  selection  of 
doctoral  students  in  a  department  of  psychology.    Dawes  was  not  concerned 
with  attempting  to  develop  a  maximally  predictive  system  nor  with 
establishing  an  ideal  criterion.    His  concern  was  to  establish  the 
utility  of  a  simple  linear  equation  to  adequately  predict  the  com- 
mittee's judgments.     He  obtained  a  multiple  correlation  of  .40  with  a 
three-variable  linear  equation.     The  variables  used  were:  overall 
undergraduate  grade  point  average,  the  total  scaled  score  on  the  apti- 
tude tests  of  the  GRE,  and  a  crude  measure  of  quality  of  the  undergrad- 
uate institution  based  on  the  selectivity  index  in  Cass  &  Birnbaum 
(1968).     Using  his  model  to  make  decisions  on  a  sample  which  had  al- 
ready been  decided  on  by  the  admissions  committee,  Dawes  was  able  to 
eliminate  55  percent  of  those  eliminated  by  the  committee  with  no 
loss  in  those  accepted  by  the  committee.     Dawes  contended  that  such 
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computerized  decisions  prove  to  be  less  capricious  and  in  the  long  run 
more  "human"  than  human  judgments. 

In  a  slightly  different  approach,  Shostak  (1970)  attempted  to 
predict  rater  policies  on  an  essay  examination  in  English  in  terms  of 
personal  and  educational  factors  about  the  raters  themselves.  He 
first  cluster  analyzed  the  examination  grades  and  then  factor  analyzed 
the  resulting  clusters  on  rater  variables.    This  was  an  ipsative  pro- 
cedure in  that  it  was  assumed  the  rater's  assigned  grades  varied  in 
accordance  with  factors  not  directly  considered  in  the  rating  process. 
In  other  words,  it  was  assumed  that  the  rater  had  a  "within  person" 
distribution  of  the  variables  of  interest  which  related  to  how  he  arrived 
at  his  ratings.    The  ipsative  approach  was  described  by  Cattell  (1968) 
and  Guilford  (1954)  in  regard  to  the  psychometric  procedure  of  factor 
analysis,  and  was  used  in  the  present  study.  ' 

The  regression  technique  used  in  the  present  study  was  based  on 
the  fixed-effects  model.     With  this  model,  there  is  no  assumption  that 
the  independent  variables  are  normally  distributed'  as  a  result,  new 
variables  can  be  generated  from  functions  of  the  independent  variables. 
Ward  (1962)  and  Bottenberg  &  Ward  (1963)  detailed  a  variety  of  computer- 
assisted  predictive  and  simulative  applications  for  the  fixed-effects 
linear  model.    A  variation  of  one  of  these  applications  was  used  in 
this  study. 

The  variation  used  herein  was  described  and  named  by  Christal 
(1968b)  the  "policy  capturing  model."    The  essential  feature  of  the 
policy  capturing  model  as  compared  to  the  usual  system  of  building  a 
prediction  equation  was  that  all  conceptually  reasonable  variables 
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and  functions  of  variables  were  tried  out  in  the  system  within  the 
limits  of  time,  money,  and  available  data.     Chris tal  reported  use  of 
the  policy  capturing  model  in  the  Air  Force  to  predict  a  board's 
decisions  about  the  grade  of  officer  personnel  each  of  3,575  jobs  in 
the  Air  Force  required.    After  examining  almost  200  possible  variables, 
Chris tal  reported  an  ability  to  predict  the  judgments  of  the  board  of 
22  colonels  with  a  9-predictor  system  which  had  a  multiple  correlation 
of  .92  with  the  board. 

Talbot  (1966)  also  used  a  fixed-effects  linear  model  to  test  the 
predictive  value  of  36  variables  in  selecting  the  39  of  195  black 
candidates  who  received  their  doctorate.    Multiple  correlation  of  pre- 
dicted successes  to  observed  successes  was  .38  for  the  full  or  unre- 
stricted model.     Of  the  36  variables,  Talbot  discovered  that  only  6 
significantly  reduced  the  residual  least  squares  value  at  the  .05 
level.     Talbot  suggested  that  all  6  of  these  predictor  variables  were 
Indicators  of  academic  achievement.     He  even  put  prior  armed  service 
in  this  category  pointing  out  that  veterans  had  usually  travelled 
more,  been  exposed  to  other  cultures,  and  were  often  more  mature  and 
less  provincial.     Talbot's  study  was  primarily  related  to  this  study 
because  of  his  use  of  the  fixed-effects  linear  model  which  was  the 
basic  statistical  tool  of  the  present  study  also. 

Hierarchical  Clustering 
Ward  &  Hook  (1963),  Chris tal  (1968a),  and  Bottenberg  &  Chris tal 
(1968)  reported  a  technique  for  grouping  criteria  in  terms  of  the 
homogeneity  of  their  associated  prediction  equations.     The  general 


procedure  was  to  develop  a  prediction  equation  for  each  rater  or 
member  of  a  judgment  or  policy  board  and  then  mathematically  combine 
the  two  equations  out  of  the  total  group  which  resulted  in  the  least 
reduction  of  overall  multiple  correlation.     This  procedure  was  con- 
tinued, reducing  the  total  number  of  equations  by  one  in  each  of  a 
series  of  steps  until  one  single  equation  was  produced.     The  advantages 
of  this  technique  were  that  maximum  predictive  qualities  were  preserved 
for  a  divergent  pattern  of  judgments ,  and  that  points  of  more  important 
differences  in  judgments  could  be  identified.     This  technique  was 
called  judgment  analysis  (JAN)  by  Chris tal  (1968a). 

Nay lor  &  Wherry  (1965)  used  the  JAN  technique  for  capturing  and 
clustering  the  policies  of  raters  from  ratings  of  hypothetical  people 
with  simulated  scores.     The  authors  maintained  that  this  procedure 
saved  time  and  money  since  otherwise  researchers  would  have  to  obtain 
large  numbers  of  people  who  had  been  quantified  on  each  dimension  of 
interest  for  the  raters  to  rate.     Naylor  and  Wherry  indicated  that  the 
use  of  hypothetical  ratees  seemed  worthwhile  statistically,  but  that 
the  actual  worth  was  still  under  investigation. 

Wherry  &  Naylor  (1966)  compared  a  system  of  clustering  which 
they  originated,  PROF,  with  the  JAN  technique.     PROF  means  a  Profile 
of  Factors  and  was  based  on  a  system  of  factor  analysis.     The  authors 
argued  that  even  though  the  JAN  technique  clustered  raters  by  the 
similarity  of  their  prediction  equations,  it  did  not  really  analyze 
their  judgments.     Their  position  was  that  capturing  policies  and 
describing  policies  are  two  different  things.     The  Profile  of  Factors 
procedure  they  outlined  involved  identifying  and  describing  factors 
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of  rater  variance  first  and  then  clustering  raters  on  the  similarity 
of  their  factors.     PROF  was  considered  by  the  authors  to  be  a  somewhat 
better  basis  for  clustering  raters  than  was  JAN,  if  the  rating  strategies 
were  to  be  meaningful  and  interpretable. 

For  the  purposes  of  the  present  study,  the  strategies  of  the 
raters  and  the  meaning  of  the  strategies  were  not  of  primary  importance 
since  prediction  was  of  all  raters  in  one  academic  area  rather  than  of 
individual  raters.     The  main  problem  was  to  identify  the  specific  vari- 
ables which  were  predictive  of  ratings  in  the  various  academic  areas.' 

Houston  (1968)  and  Houston  &  Roscoe  (1968)  asked  20  faculty 
judges  to  rate  each  of  231  recent  graduates  from  graduate  education 
programs  on  the  basis  of  21  variables.     The  former  students  were  rated 
on  a  variety  of  continuous  and  categorical  variables  without  the  faculty 
knowing  whom  they  were  rating.     Using  these  faculty  ratings  as  criter- 
ion, a  prediction  system  was  developed  with  the  JAN  technique  yield- 
ing a  multiple  correlation  of  .93.     Houston  &  Roscoe  (1968)  then  asked 
the  faculty  to  rate  the  same  students  by  name  only.     Using  this  rating 
as  criterion,  the  same  variables  were  employed  to  produce  another  pre- 
diction system  by  means  of  the  JAN  technique.     This  particular  approach 
the  authors  called  the  ipsative  variation;  the  multiple  correlation 
obtained  was  .42.     Houston  and  Roscoe  commented  that  probably  the 
ipsative  variation  could  not  be  considered  to  have  captured  the  policy 
of  the  raters  in  terms  of  the  variables,  but  was  simply  a  statement 
of  the  amount  of  relationship  between  the  variables  and  the  ipsative 
JAN  criterion. 

In  another  study  using  the  JAN  technique,  Houston  &  Bentzen 
(1969)  attempted  to  capture  the  teacher  evaluation  policy  of  a  team 


of  evaluators.    Their  approach  was  to  use  a  modified  form  of  the  JAN 
technique  to  express  the  joint  policy  of  the  team.     They  then  tested 
the  predictive  contributions  of  one  group  of  variables  termed  teacher 
characteristics  and  another  group  termed  pupil  characteristics. 
Houston  and  Bentzen  reported  that  a  policy  had  been  captured  and  that 
the  group  of  variables  called  teacher  characteristics  contributed 
significantly  to  prediction.  ■  . 

Williams,  Gab,  &  Lindem  (1969)  tried  to  capture  the  policy  of 
a  university  department  concerning  admissions  to  the  doctoral  program. 
They  had  12  graduate  faculty  members  sort  the  profiles  of  33  graduated 
doctoral  students  into  seven  different  sorts   (using  a  Q-sort  technique). 
The  profiles  consisted  of  14  variables  including:    MAT  score,  age,  and 
GRE  scores.     The  seven  sorts  were  assigned  numerical  weightings  from 
1  to  7  which  became  the  criterion  variable  for  developing  a  prediction 
equation  for  each  faculty  member.     Clustering  of  the  equations  by  their 
homogeneity  was  then  accomplished  by  use  of  the  JAN  technique.  Results 
of  the  clustering  indicated  that  there  were  at  least  two  and  possibly 
three  groups  of  rating  systems  among  the  faculty  in  making  admissions 
decisions.     Williams,  Gab,  and  Lindem  reported  that  the  most  important 
aspect  of  this  study  was  that  it  revealed  to  the  faculty  their  percep- 
tions about  the  admissions  variables. 

The  final  example  of  a  use  of  the  JAN  technique  is  a  study  con- 
ducted by  Littleton  (1970).     This  study  attempted  to  capture  the 
policy  of  elementary  principals  in  regard  to  their  predisposition  to 
try  selected  innovations.     Two  subsets  of  variables  were  identified. 
One  group  was  variables  associated  with  the  perceived  characteristics 
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of  the  innovation  being  considered.    The  other  set  of  variables  was 
associated  with  characteristics  of  the  power  structure  into  which  the 
Innovation  was  being  introduced.     The  basic  findings  were  that  the 
policies  of  the  principals  could  be  captured  and  that  both  groups  of 
variables  contributed  significantly  to  prediction. 

The  predictive  qualities  of  three  groups  or  subsets  of  variables 
were  tested  in  the  present  study.     Since  the  overall  prediction  equation 
to  which  the  predictive  efficiency  of  each  subset  was  tested  was  devel- 
oped by  the  JAN  technique,  Littleton's  study  was  quite  similar  metho- 
dologically to  this  study. 

Summary 

Summarizing  the  literature,  the  need  for  accountability  in  edu- 
cation, including  graduate  education,  seemed  apparent.  Accountability 
in  general  implies  some  kind  of  system  for  reconciling  means  with  ends 
in  education.    An  essential  requirement  of  virtually  all  approaches 
to  accountability  reviewed  was  an  information  system  which  can  indi- 
cate relational  patterns  between  accountability  measures. 

One  statistical  technique  which  appeared  sufficient  for  inte- 
grating measures  of  accountability  into  meaningful  relationships  was 
regression  analysis.     The  fixed-effects  model  in  regression  analysis 
seemed  particularly  appropriate  since  it  permitted  functions  of 
variables  to  be  used  as  variables.     The  "prediction  of  ratings" 
studies  reviewed  tended  to  support  the  contention  that  regression  analysis 
was  adequate  for  relating  a  large  number  of  accountability  measures  with 
acceptable  results.  j 
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The  studies  reviewed  in  relation  to  the  hierarchical  cluster- 
ing procedure  used  in  the  present  study  indicated  that  JAN  was  a 
powerful  and  versatile  statistical  tool.     Though  diverse  in  subject, 
the  reviewed  studies  showed  the  capability  of  the  JAN  technique  to 
cluster  criteria  by  their  homogeneity  while  retaining  maximum  predic- 
tive efficiency.  ' 


CHAPTER  III 


PROCEDURES 

The  criterion  variables  for  this  study  were  obtained  from  past 
administrations  of  the  doctoral  written  general  qualifying  examina- 
tion as  recorded  in  the  files  of  the  University  of  Florida  Graduate 
Studies  in  Education  Office.     Therefore,  the  purposes,  practices  of 
administration,  and  grading  policies  for  conducting  the  examination 
have  a  bearing  on  the  study  criteria.     So  that  the  nature  of  the 
criterion  variables  might  be  better  understood,  an  explanation  of 
the  general  qualifying  examination  is  presented  below.  Additionally, 
since  i.nformation  from  student  files  in  the  Graduate  Studies  Office 
formed  the  basis  for  the  predictor  variables  of  this  study,  the  specif 
ic  types  of  information  used  are  given  in  three  general  categories. 
Also,  the  population  of  doctoral  students  studied  is  described.  Next, 
the  procedures  in  analyzing  the  data  are  discussed.    Finally,  a 
chapter  summary  is  given. 

Explanation  of  the  General  Qualifying  Examination 
The  general  written  qualifying  examination  has  been  an  integral 
part  of  the  doctor  of  education  program  since  the  degree  was  first 
offered  in  1949.     Through  the  spring  of  1969,  the  examination  was  ad- 
ministered several  times  a  year  on  a  cross-departmental  or  college- 
wide  basis,  with  purposes,  procedures,  and  grading  policies  constant 
through  time . 
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Purposes 

In  a  previously  cited  study,  Molbert  (1960)  reported  the  purpose 
of  the  examination  was  to  determine  the  level  of  development  of  certain 
competencies  in  the  doctoral  candidate  "...  both  for  the  professional 
status  the  degree  represents  and  for  the  realization  of  the  individual's 
potentialities"  (p.  4).    Her  study  includes  an  instruction  to  the 
faculty  members  writing  and  grading  the  actual  examination  to  write  and 
grade  a  question  representative  of  knowledge  which  "...  could  reason- 
ably be  expected  of  any  graduate  student  at  the  doctoral  level"  (p.  291) 

Practices  of  Administration 

The  complete  procedures  for  administration  of  the  general  quali- 
fying examination  are  presented  in  Appendix  A.    Three  procedural  matters 
however,  are  included  here  for  general  understanding. 

First,  the  examination  was  conducted  on  a  completely  anonymous 
basis.     Identity  of  the  faculty  members  who  prepared  and  graded  the 
questions  at  each  testing  was  kept  completely  confidential,  before, 
during,  and  after  actual  administration  of  the  examination.  Also, 
identity  of  the  students  was  linked  to  their  examination  papers  only 
by  a  special  code  number  which  was  not  available  to  the  graders. 

Second,  for  the  time  period  of  interest  in  this  study,  the  quali- 
fying examination  included  questions  in  five  academic  areas  of  educa- 
tion.    These  areas  were:     educational  administration,  psychological 
foundations  of  education  and  personnel  services ,  sociological  founda- 
tions of  education,  curriculum  and  instruction,  and  philosophical  and 
historical  foundations  of  education. 
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Third,  at  each  offering  of  the  examination,  a  group  of  ten  faculty 
members  was  selected  to  write  and  grade  the  examination  questions. 
These  ten  faculty  were  chosen  in  five  sets  of  two  with  each  set  having 
academic  competency  in  one  of  the  five  areas  covered  on  the  examina- 
tion.    Each  set  of  two  faculty  members  jointly  prepared  an  examination 
question  in  their  academic  area.     Later  these  same  two  faculty  members 
graded  their  joint  question  independently,  observing  the  general  grad- 
ing policies  below.  '      ,1*,  ■« 

Grading  Policies 

In  arriving  at  a  grade  for  a  question,  an  individual  faculty  mem- 
ber assigned  a  score  ranging  from  1  to  5  on  each  of  five  grading  criter- 
ia, then  averaged  these  scores  to  produce  the  grade  score.  These  in- 
dividual faculty  grade  scores  were  the  criterion  variables  or  variables 
of  interest  for  this  study.     For  more  detailed  information  regarding 
grading  policies  see  Appendices  A  and  B. 

Student  Information 
As  indicated  in  the  introduction  to  this  section,  information 
from  the  students'  files  formed  the  basis  for  the  predictor  variables 
in  this  study.     Presented  below  are  the  three  informational  sources 
of  data  and  the  predictor  variables  derived  from  each  source.  Pre- 
dictor variables  collected  from  each  informational  source  were  of  two 
forms:     those  obtained  directly  from  the  informational  source,  and 
those  formulated  by  combinations  of  directly  obtained  variables. 
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Pre-Admission  Information 

Information  available  about  a  student  prior  to  admission  for 
doctoral  study  constituted  this  source  of  data.     The  predictor  vari- 
ables obtained  directly  from  this  source  are  shown  in  Table  1.  The 
reason  for  selecting  these  predictor  variables  for  investigation  was 
because  they  generally  represented  consistently  available  informa- 
tion which,  it  was  believed,  might  have  some  predictive  relationship 
to  the  criterion  grades. 

Various  combinations  of  the  directly  obtained  variables  above 
were  used  to  formulate  additional  predictor  variables.    The  result- 
ing combination  variables  are  shown  in  Appendix  C  as  functions  of  the 
directly  obtained  variables.     Squares  of  variables  were  introduced 
where  it  was  suspected  second  order  curvilinear  relationships  might 
exist  between  predictors  and  the  criteria.     Products  of  variables 
were  included  in  order  to  generate  interaction  predictors  where  such 
seemed  reasonably  appropriate.  ' 

Post -Admission  Information 

Information  generated  after  admission  for  doctor  study  was 
this  source  of  data.     Predictor  variables  collected  directly  from 
this  source  are  presented  under  the  appropriate  heading  in  Table  1. 
Additional  variables  produced  from  functions  of  the  directly  obtained 
variables  can  be  seen  in  Appendix  C.     These  post-admission  variables 
were  included  for  study  for  the  same  reasons  as  the  pre-admission 
variables. 
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Graduate  Program  Information 

Information  generated  as  a  result  of  the  total  graduate  study 
program  comprised  this  source  of  data.     Predictor  variables  based  on 
these  data  are  presented  in  Table  1  and  Appendix  C.     The  rationale 
for  inclusion  of  variables  here  was  the  same  as  for  those  from  the  two 
preceding  sources. 

Study  Population 

The  population  for  this  study  was  all  doctoral  students  in  the 
University  of  Florida  College  of  Education  who  took  the  general  quali- 
fying examination  during  the  years  1963  through  1969.     There  were  two 
changes  in  qualifying  examination  procedures  referred  to  previously 
which  limited  the  population  to  these  years.     First,  a  minor  change 
in  grading  policy  was  effected  in  1963.     The  change  resulted  in 
slightly  higher  grades  on  the  average  from  1963  on,  making  comparisons 
with  prior  grades  difficult.     Second,  at  the  end  of  1969  administra- 
tion became  the  responsibility  of  the  individual  departments. 

No  sampling  of  the  study  population  was  necessary.     The  entire 
population  of  doctoral  students  who  took  the  qualifying  examination 
during  the  years  1963  -  1969  was  used  in  this  study.     The  total  number 
of  students  was  357.     General  characteristics  of  the  population  are 
summarized  in  Appendix  D. 

Analysis  of  the  Data 
Analysis  of  the  data  is  considered  in  two  parts.     First  the  study 
design  is  given.     Then  the  actual  data  treatment  is  presented. 
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Study  Design 

As  was  mentioned  in  the  section  explaining  the  general  qualify- 
ing examination,  the  individual  faculty  grade  scores  were  the  criter- 
ion variables  for  the  study.     Comparisons  of  these  scores  between  the 
five  academic  areas  of  the  qualifying  examination  revealed  a  rather 
low  intercor relation  (Table  2).    Because  of  the  low  intercorrelation, 
five  separate  prediction  equations  were  developed,  one  for  each  academic 
area.    Next,  the  equations  were  clustered  in  terms  of  their  predictive 
agreement.    The  three  steps  used  to  accomplish  this  general  design 
follow. 

The  first  step  in  the  study  design  was  to  produce  five  linear 
regression  equations  to  predict  the  grades  received  in  the  five  aca- 
demic areas  included  on  the  qualifying  examination.     Since  two  faculty 
members  in  each  area  independently  graded  the  students  tested,  the 
effective  number  of  criterion  variables  for  each  area  was  twice  the 
number  of  students  tested.     The  five  regression  equations  were  devel- 
oped from  the  predictor  variables  by  a  stepwise  regression  procedure 
in  accordance  with  the  policy  capturing  model  described  by  Christal 
(1968b). 

The  second  step  in  the  study  design  was  to  cluster  the  five 
prediction  equations  to  produce  a  single  best  equation  for  represent- 
ing composite  success  on  the  examination.     The  technique  employed 
grouped  the  equations  in  terms  of  their  homogeneity  while  retaining 
maximum  predictive  efficiency.    Additionally,  this  clustering  process 
revealed  patterns  of  agreement  in  outcome  expectations  between  the 
academic  areas. 
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The  third  step  of  the  study  design  was  to  conduct  three  tests 
of  significance  for  the  predictive  qualities  of  three  subsets  of  vari- 
ables.    The  three  subsets  were:     pre-admission  variables,  post-admission 
variables,  and  program  variables. 

Data  Treatment 

Computation  involved  in  all  multiple  regression  analyses  was  per- 
formed by  an  IBM  360  computer.     The  computer  was  programmed  according 
to  the  biomedical  stepwise  regression  procedure,  BMD02R  (Efroymsen, 
1960). 

Clustering  of  the  regression  equations  by  their  homogeneity  was 
accomplished  by  a  hierarchical  grouping  technique,  JAN  (Bottenberg  & 
Christal, 1968a;  Chris tal, 1968b) .     Tests  of  significance  in  regression 
of  pre-admission,  post-admission,  and  program  variables  were  calculated 
in  accordance  with  Bottenberg  &  Ward  (1963). 

Summary 

In  summary,  the  criteria  for  this  study  were  scores  in  five  dif- 
ferent academic  areas  of  the  qualifying  examination.     The  examination 
procedures  were  rigorous  and  the  purposes  reflected  a  concern  for  pro- 
fessional standards.     The  independent  variables  were  drawn  from  three 
classifications  of  information:     that  available  at  the  time  of  admis- 
sion to  doctoral  study,  that  generated  after  admission  to  doctoral 
study,  and  that  specifically  related  to  the  overall  doctoral  program 
of  courses.     The  study  population  was  357  doctoral  students  who  took 
the  qualifying  examination  from  1963  to  1969.     Analysis  of  the  data 
involved  developing,  by  a  stepwise  computer  program,  five  regression 
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equations,  one  to  predict  each  of  the  academic  areas  of  the  qualify- 
ing examination.     These  equations  were  then  clustered  by  a  hierarchical 
grouping  procedure  to  reveal  patterns  of  predictive  agreement  and  to 
yield  a  single  "best"  prediction  equation.     Finally,  the  predictive 
significance  of  variables  from  each  of  the  information  sources  dis- 
cussed above  was  tested. 


CHAPTER  IV 
RESULTS 

This  chapter  is  composed  of  three  parts.    In  the  first  part  five 
multiple  regression  equations  are  presented  and  explained.     Each  of 
these  equations  was  a  prediction  system  for  the  outcome  expectations 
of  one  academic  area  as  reflected  in  the  scores  on  that  portion  of 
the  qualifying  examination.     Although  multiple  regression  equations 
are  useful  because  of  the  predictive  qualities  of  properly  weighted 
combinations  of  variables,  they  are  not  very  helpful  for  describing 
the  factors  which  appear  to  underlie  measures  of  outcome  (Wherry  & 
Kaylor,  1966;  Bottenberg  &  Ward, 1963).     For  this  reason  the  rela- 
tionship of  some  individual  correlates  of  the  outcome  variable  with 
the  variables  in  regression  was     also  examined. 

The  second  section  contains  the  results  of  a  hierarchical 
clustering  procedure  which  grouped  the  five  equations  in  terms  of 
their  homogeneity.     Predictive  qualities  of  each  of  three  subsets 
of  variables,  pre-admission  variables,  post-admission  variables, 
and  program  variables,  are  discussed  in  part  three  of  this  chapter. 

Multiple  Regression  of  the  Five  Academic  Areas 
As  was  previously  mentioned,  the  multiple  regression  procedure 
used  was  a  stepwise  linear  computer  program  (Efroymsen,  1960).  In 
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this  program  an  additional  variable  was  added  to  the  regression  system 
in  a  stepwise  manner  until  either  F  values  or  tolerances  of  remaining 
variables  failed  to  meet  the  specified  requirements  to  enter  the  sys- 
tem.   The  variable  added  at  each  step  was  the  one  which  resulted  in 
the  greatest  reduction  in  residual  sum  of  squares.     Since  the  number 
of  variables  in  the  present  study  was  quite  large,  the  number  of  steps 
in  the  regression  analysis  was  commensurately  large. 

In  order  to  keep  the  presentation  of  data  to  a  more  useful  level. 
Appendices  F  -  I  show  regression  equation  parameters  for  each  area  in 
steps  one  through  five  and  step  ten  only.     Stepwise  presentations  of 
variables  added  are  limited  to  the  first  five  steps  while  increases  in 
squared  multiple  correlation,  R^,  are  shown  in  the  first  five  steps 
plus  step  ten  and  the  final  step  in  Tables  3,  5,  7,  9,  and  11.  The 
criterion  variable  in  each  of  the  following  multiple  regression  equa- 
tions was  the  score  on  the  qualifying  examination  in  the  academic 
area  being  shown. 

Multiple  Regression  of  Educational  Administration 

2 

As  can  be  seen  in  Table  3  the  maximum  attained  multiple  R  for 
expressing  outcome  scores  in  educational  administration  was  .19.  This 
value  was  reached  in  72  steps  and  indicated  that  approximately  19  per- 
cent of  the  variation  in  the  qualifying  examination  scores  for  this 
academic  area  was  accounted  for  in  terms  of  the  independent  variables. 
The  maximum  predictability  of  educational  administration  relative 
to  the  other  areas  on  the  qualifying  examination  can  be  viewed  in 
Appendix  E. 
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Comparison  of  the  data  in  Tables  3,  5,  7,  9,  and  11  indicated 

that  a  reasonable  equation  for  the  multiple  prediction  of  outcome  in 

each  academic  area  of  the  qualifying  examination  was  attained  in  three 

2 

steps  of  regression.    Beyond  the  third  step,  increases  in  R    were  re- 
duced to  .01  in  all  cases.     The  three-variable  equation  for  educational 
administration  was  written  from  Appendix  F,  Step  3,  as  follows: 
Educational  Administration  Score  = 

+1.0207  ^. 

+  .0012     (Advanced  Test  of  GRE) 

+  .2528     (Major  Department  Educational  Administration) 
+  .4587     (Post-Admission  Graduate  Honor  Point  Average) 
Interpretation  of  regression  equations  must  be  approached  with 
caution  because  of  various  limiting  factors.     These  factors  are  dis- 
cussed in  Chapter  V.     It  is  possible,  however,   to  say  that,  although 
this  equation  did  not  represent  a  unique  three-variable  combination 
relationship  with  the  criterion,  it  did  represent  a  maximum  relation- 
ship.    That  is  to  say,  the  multiple  correlation,  R,  was  the  maximum 
possible  between  the  educational  administration  outcome  measure  and 
any  combination  of  three  of  the  investigated  measures  of  selecting 
and  training  doctoral  students. 

Thus  the  equation  above  was  useful  in  that  it  gave  an  optimum 
three-variable  combination  for  prediction  of  the  criterion  variable. 
It  was  incorrect,  however,   to  assume  that  the  variables  included  in 
this  equation  were  necessarily  those  most  highly  correlated  on  an  in- 
dividual basis  with  the  outcome  measure.     Each  independent  variable  in 
this  equation  was  more  realistically  viewed  as  a  representative  of  a 
complex  of  variables  or  a  factor. 
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This  was  so  because  the  basis  for  selection  of  new  variables  at 
each  step  of  regression  was  the  variable  which  had  the  highest  partial 
correlation  with  the  dependent  variable  partialed  on  the  variables 
which  had  already  been  added.     Thus,  the  variables  in  the  equation  had 
a  relatively  low  intercorrelation,  but  may  each  be  conceived  of  as 
representing  other  potential  variables  with  which  their  intercorrela- 
tion was  relatively  high  (Guilford,  1956). 

In  order  then  to  give  a  broader  perspective  to  the  variables  in 
the  equation,  their  intercorrelation  with  the  ten  highest  criterion 
correlates  from  among  all  the  study  variables    is    shown  in  Table  4. 
The  greatest  zero  order  correlation  in  each  row  was  indicated  so  that 
the  ten  row  correlates  were  clustered  with  the  three  equation  variables 
by  column. 

Examination  of  the  data  thusly  tabled  reflected  generally  that 
a  combination  of  three  factors  appeared  to  have  been  related  to  suc- 
cess on  the  outcome  measure  in  educational  administration.  These 
factors  were: 

(a)  grades  in  doctoral  study, 

(b)  orientation  toward  the  ideas  of  the 
Educational  Administration  Department,  and 

(c)  prior  academic  achievement  as  indicated  by  GRE 
score. 

Multiple  Regression  of  Curriculum  and  Instruction 

By  referring  to  Table  5,  it  can  be  seen  that  a  maximum  of 
.20  was  reached  in  predicting  the  curriculum  and  instruction  portion 
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of  the  qualifying  examination.     Thus,  approximately  20  percent  of  the 
variation  in  the  outcome  scores  was  accounted  for  in  79  steps.  Appen- 
dix E  shows  the  predictability  of  curriculum  and  instruction  relative 
to  the  other  academic  areas  on  the  examination. 

From  the  prior  comparison  of  Tables  3,  5,  7,  9,  and  11,  three- 
variable  equations  were  selected  for  exposition  in  all  academic  areas. 
The  three-variable  equation  for  the  curriculum  and  instruction  area 
was  written  from  Appendix  G,  Step  3,  as  follows:  - 

*. 

Curriculum  and  Instruction  Score  = 
+  .1810 

+  .0008  (Verbal  Aptitude  Test  of  GRE) 
+  .2090  (Major  Department  Curriculum  and  Instruction) 
+  .7111  (Post-Admission  Graduate  Honor  Point  Average) 

The  variables  in  this  regression  equation  were  given  more  defini- 
tion by  referring  to  Table  6.     This  table  shows  the  intercorrelation 
between  the  three  variables  in  regression  and  the  ten  study  variables 
which  correlated  most  highly  with  the  criterion.     By  observing  the 
variables  marked  in  each  column,  a  crude  indication  of  the  factors 
represented  by  the  equation  variables  at  the  top  of  each  column  was 
obtained.    Broadly  speaking,  a  combination  of  three  selection  and 
training  factors  seemed  to  have  been  related  to  the  outcome  measure 
in  Curriculum  and  Instruction.     The  factors  were: 

(a)     grades  in  doctoral  study, 
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(b)  orientation  toward  the  ideas  of  the 
Curriculum  and  Instruction  Department,  and 

(c)  verbal  aptitude  as  indicated  most  effective- 
ly by  GRE  score. 

Multiple  Regression  of  Historical  and  Philosophical  Foundations 

Table  7  shows  that  approximately  29  percent  of  the  variation  in 
outcome  scores  for  this  academic  area  was  accounted  for  in  89  steps 
by  a  combination  of  the  study  variables.    Appendix  E  allows  a  compari-  ^ 
son  of  this  academic  area  with  the  other  areas  in  the  degree  to  which 
multiple  prediction  was  possible  from  these  variables. 

As  was  previously  mentioned,  based  on  a  comparative  examination 
of  Tables  3,  5,  7,  9,  and  11,  a  three-variable  equation  was  selected 
for  further  discussion.     From  Appendix  H,  Step  3,  the  three-variable 
equation  for  the  multiple  prediction  of  the  historical  and  philosophi- 
cal portion  of  the  qualifying  examination  was  as  follows: 

Historical  and  Philosophical  Foundations  Score  = 
-1.5253 

+  .0017  (Verbal  Aptitude  Test  of  GRE) 

+1.0864  (Pos t- Admission  Graduate  Honor  Point  Average) 

-  .0001  (Square  of  Age) 

Additional  meaning  was  given  to  the  equation  variables  by  perusing 
Table  8.     Speaking  generally,  a  combination  of  the  following  three 
factors  appeared  to  have  been  related  to  the  outcome  measure  in  this 
academic  area: 
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^   (a)    grades  in  doctoral  study, 

(b)  verbal  aptitude  as  indicated  most 
effectively  by  GRE  score,  and 

(c)  age. 

Two  points  should  be  noted  in  regard  to  the  age  factor.    First,  younger 
students  performed  better  than  older  ones.    Second,  a  quadratic  func- 
tion was  a  better  statement  than  a  linear  for  expressing  the  relation- 
ship  between  age  and  performance. 

Multiple  Regression  of  Psychological  Foundations 

The  data  in  Table  9  show  than  an  R    of  .22  was  obtained  in  76 
steps  in  the  multiple  prediction  of  the  psychological  foundations 
section  of  the  qualifying  examination.     This  value  was  the  maximum 
possible  with  the  study  variables  and  indicated  that  they  accounted 
for  about  22  percent  of  the  variation  in  the  criterion  scores. 
Appendix  E  compares  the  degree  of  predictability  of  this  academic  area 
from  the  study  variables  with  the  predictability  of  the  other  areas  of 
the  qualifying  examination. 

In  keeping  with  an  earlier  assessment  of  Tables  3,  5,  7,  9,  and 
11,  a  three-variable  prediction  system  for  outcome  scores  in  psycho- 
logical foundations  is  shown.    From  Appendix  I,  Step  3,  a  three-variable 
equation  for  the  multiple  prediction  of  this  area  was  written: 

Psychological  Foundations  Score  = 
+  .9935 

-  .1205  (Graduation  Rate  Factor  for  Institu- 
tion where  Freshman  English  Taken) 
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+  .2875  (Honor  Point  Average  in  Social  Foundations  Courses) 
+  .5553  (Post-Admission  Graduate  Honor  Point  Average) 

The  meaning  of  the  variables  in  this  equation  was  enhanced  by 
referring  to  Table  10.    From  the  measures  of  training  and  selecting  ■ 
doctoral  students  included  in  the  present  study,  it  appeared  that  a 
combination  of  three  general  factors  related  to  success  on  the  psycho- 
logical foundations  outcome  measure.    These  factors  were: 

V  ./-lite.  - 

(a)  grades  in  doctoral  study, 

(b)  performance  in  social  foundations  courses,  ' 
and 

(c)  the  type  of  institution  attended  for  early 
years  of  undergraduate  study. 

Two  observations  were  in  order  regarding  the  nature  of  this  last 
factor.     In  the  first  place,  there  was  a  high  correlation  (.74)  between 
the  graduation  rate  indexes  for  the  first  two  and  last  two  years  of 
undergraduate  study.     The  high  correlation  between  the  two  was  probably 
because  many  students  stayed  at  the  same  institution  for  four  years. 
Secondly,  it  was  noted  that  the  graduation  rate  index  was  negatively 
related  to  the  criterion  for  this  area.     This  meant  that  students  from 
undergraduate  institutions  with  low  graduation  rates  did  less  well  on 
the  psychological  foundations  criterion. 

Multiple  Regression  of  Social  Foundations 

It  can  be  seen  from  Table  11  that  approximately  20  percent  of 
the  score  variation  in  the  social  foundations  section  of  the  qualifying 


-59- 


o 
.c 
o 

s  s 

<4-l  -r-l 
U 

m  oJ 
0)  C 

4J  -i-l 
1-1  TO 

0)  X 

»-i  w 
)-< 

O  60 

1-1 


(J)  rt 

4-1  3 
•W  O" 

c 

o 


ca 

J-i 


> 

CO 

o  o 

a)  C 
n  d 
ft,  o 

m 
o 


o 

60 
O 


<U 
VI 

o 
u 
u 
to 
*i 

5 


1 

<u 

•H 

to 

u 

4-) 

0) 

TO 

tn 

TO 

oa 

In 

H 

c 

Q) 

o 

>-i 

^4 

CO 

•H 

o 

(1) 

•H 

4-1 

r-H 

nj 

3 

!-i 

c 

T5 

O 

C 

<^ 

u 

o 

o 

•r4 

c 

4-1 

3 

B 

CO 
0)1 

t-ll 

•° 

CC 

1-1 

CO 

> 
o 

•"-•l 
-^1 

C8 
^1 

w 

•1-J 

col 

CO 
Q) 

SI 


to 

r-(  0) 
C3  CO 

•H  1-1 

4J    O  3 

coo 

•H  CO  O 

O 


1-4 
O 

c 
o 


C  CO 
1-1  C 
o| 

Q)  ■ 
60  . 
CO  TO 
l-l  T) 
CD  Cl 
>  3 

<;  o 
1^1 


c 

u 

cul 

o 

o 

•H 

a 

rt| 

CO 

o 

1-1 

CO 

X 

0) 

•r-l 

> 

s 

0) 

< 

JJ 

<Ji 

TO 

4-1 

1 

3 

c 

4J 

T) 

•H 

CO 

TO 

O 

o 

1-1 

o 

TO  CO 

O  C 

•H  O 

60  1-1 

O  4J 

r-l  TO 

O  tJ 

^  c 

O  3 

CO  (i4 
Pi 


col 

O 

1-1  rt| 

1-1  ' 

CU  0) 

4J  Ul 

•r-l  M 

IJ  O 

o  o 


00 


o 
o 


I 

c 

3 
O 

P4 


TO  4-> 

O  -^4 

•r^  u 

60  O 
O 

.-I  CO 

O  C 

o 

O  1-1 

CO  TO 
Pi  T) 


\0 

o 
I 


CM 


TO 

O 

O 


C 

o 

CO 
CO 


B  <u 

<;  TO 

I  3 

CO  TO 


0) 
60 
TO 
Vi 

<1> 


4J 

c 

•H 

O  1-1  O 
Pi  O  Pi 


in 
o 


CO 

o 
o 


CM 


CM 


<U 
60 

TO  I 

11  TO 

<U  13 

^  § 

4J  [14 
S 

•H  r-l 

O  TO 

Pi  O 

O 

U  O 

o  c/) 
c 

2.  c 

33 


O 

o 


1! 


CM 


* 
* 

0^ 


0) 
Q) 
CO 
H 
3 
O 
O 


60 
TO 

V4  . 
d)  TO 

!5  " 

<  "1 
60 

4J  O  to 
C  r-<  C 

1^  o  o 
o  x:  -ri 

P4    CJ  4-1 

>,  TO 
11  CO  t3 
O  Pi  C 
C  3 
O  C  O 
X  -r^ 


(0 
O 
O 


in 
o 


\0 

o 
I 


CO 


o  <u  <u 

o  <u 
TO  x:  H 

C  CO 
1-1 


0) 


4J  O 
TO 
PJ 


60 

c 
w 


3 

o  - 

1-1  4-1  C! 

4J  CO  TO 

TO    C  B 

3  H  x: 

T)  CO 

TO   li  <U 

M    O  li 

O 


-60- 


I 


O 


o 
o 


00 
o 


CM 

o 


10 

in 

CM 


CM 


to 

r-l 
I 


o 

CM 


St 

O 
I 


t 

c^ 


CM 


t 

CO 

o 


00 
o 
I 


to 

e 

1-1 

<u 


I 


t 


vO 


0) 
4J 
4J 
•i-l 

E 
o 

c 

0) 
0) 

O 
> 

to 


u 

1 

a 

to 

o 

4J 

T-l 

4J 

c 

c 

iH 

(U 

o 

to 

1 

0) 

•H 

CO 

M 

o 

to 

u 

u 

1 

60 

B 

B 

p4 

o 

a) 

to 

o 

C 

to 

CO 

•I-l 

O 

4-1 

I-l 

tH 

•I-l 

u 

cu 

o 

O 

o 

0) 

c 

to 

o 

c 

B 

to 

a) 

4J 

o 

o 

•l-l 

to 

I-l 

1-1 

Q 

4J 

to 

u 

C 

o 

T3 

X 

to 

3 

(U 

to 

0} 

OS 

4.1 

3 

o 

3 

w 

c 

O 

60 

1 

aj 

tu 

P 

CO 

4J 

o 

O 

< 

x; 

H 

c 

4J 

i-i 

to 

u 

CO 

60 

c 

•H 

<D 

60 

o 

•i-l 

to 

to 

o 

c 

•H 

J-i 

C 

U-l 

4J 

•T-l 

•o 

1-1 

U 

CJ 

c 

I-l 

1-1 

O 

to 

o 

o 

O 

<u 

4J 

0) 

u 

B 

o 

0) 

to 

>, 

(ii 

O 

to 

u 

to 

C 

to 

(SI 

XI 

60 

1-1 

y-l 

3 

4-1 

tu 

c 

3 

M 

PQ 

< 

to 

•a 

-I-l 

c 

U 

-a 

to 

u 

to 

0) 

I-l 

0) 

I-l 

o 

o 

u 

V-i 

to 

> 

to 

)^ 

60 

60 

ij 

0) 

B 

CO 

•r-( 

c 

u 

3 

O 

c 

I-l 

to 

> 

0) 

3 

4J 

o 

c 

CO 

o" 

<: 

>4-( 

Q 

PL, 

d 

< 

ceS 

O" 

to 

•I-l 

o 
to 

to 

c 

•H 

o 
u 


to 

•u  C 

c  o 


-61- 


O  0) 


c 


0) 

w 

0) 

u  (as 


o 
a 
a 


§ 
1-1 

m 

m 

d)  CO 

K  C 
00  o 

di  T-l 

OS  4J 

CC 

(1)  t3 
.H  C 

CU  3 
1^  O 

•U 

5 

S  « 

0)  u 
Si  o 

•U  CO 

o 


«M 


a 

•H 

CO 
0) 

to 
cd 

0) 

u 
o 
c 

H 


CM 
PS 


OS 


00 


O 


O 


in 

CM 


4J 

a 
1-1 
o 

o 
c 
o 

01 

iJ 
rt 
3 

-a 

M 

o 

c 
o 
1-1 

CO 
CO 

1-1 

B 

■a 
I 

4-1 

CO 

o 
a, 


60 
CO 


CNI 


o 
1-^ 


CM 
O 


00 
o 


00 
CM 


I 

u  o 

O  D 
«4-l  1-1 
0) 

(U  CO 
00 

CO  <U 


U 

<U 


o 

P-l  CO 


Q 


4) 


O 


CO  U 

3  CO 

no 

CO  1-1 

&0 

•a  •u 
c 


o 
1-1 

5  c 

•H  <U 
4J  ^ 

CO  CO 

C  H 
M 

0)  CO 

x; 

60 

M  c 
o  w 

g 

)-i  cO 

o  a 

O  CO 
CO  <U 

>> 

4J  0) 
•rH  1-1 

>  (U 

•r-i 


CM 


C3> 

o 


cn 


in 


00 
in 


CM 


O 


o 
o 


1-1  CM 

o  o 


00 

o 


o 


CM 


O 
CM 


t 

o 

CO 


.-4 

CO 


%  t 
in  m 

CO  <t 


4J 

CO 

3 

■a 

CO 

u 

60 

o 

<4-l 

c 

o 

JJ 

c 

CO 

•H 

o 

H 

CO 

<U 

0) 

60 

13 

c 

3 

CO 

4-1 

o 

4J 

H-l 

<: 

o 

i-H 

!-( 

CO 

Q) 

3 

XI 

4J 

CO 

> 

B 
5 


CO 
C 

o 
1-1 

4J 

CO 
O 
3 
-O 

W  CO 

C  CO 

•r4  1-1 

3 

CO  O 
4-1  CJ 


o 


c 
o 


CO 

4-1  U 
•r^  4-1 
t-l  CO 

CO  -i-l 

3 
O" 


CO 

vO  -i-l 


CO 


-62- 

examination  was  accounted  for  in  76  regression  steps.    A  comparison 

of  the  predictability  of  this  academic  area  relative  to  the  other 

four  areas  is  shown  in  Appendix  E.     As  can  be  noted  in  Table  11  only 

2 

small  increases  (less  than  .01)  in  R    resulted  from  the  addition  of 
each  variable  past  the  third.     Therefore,  a  three-variable  equation 
is  discussed.    The  following  three-variable  equation  was  written  from 
Appendix  J,  Step  3: 

Social  Foundations  Score  =  '    \^  « 

+  .2724 

+  .0007  (Verbal  Aptitude  Test  of  GRE) 
+  .6616  (Post-Admission  Graduate  Honor  Point  Average) 
+  .0142  (Undergraduate  Honor  Point  Average  for  the 
First  Two  Years  Times  the  Selectivity 
Factor  for  the  Institution  Where  Fresh- 
man English  Taken) 

In  order  to  provide  a  somewhat  rough  interpretation  of  the  vari- 
ables in  the  equation  above,  Table  12  was  consulted.     The  intercorrela- 
tions  in  this  table  give  a  suggestion  of  possible  factors  represented 
by  the  variables  in  regression.     Additional  correlates  of  each  equa- 
tion variable  were  indicated  in  the  appropriate  column. 

In  terms  of  the  investigated  measures  of  selecting  and  training 
doctoral  students,  a  combination  of  three  factors  was  related  to  the 
social  foundations  outcome  measure.     These  three  factors  were: 

(a)    grades  in  doctoral  study, 
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(b)  undergraduate  grades,  especially  when 
weighted  by  admissions  selectivity  of  the 
undergraduate  institution,  and 

(c)  verbal  aptitude  as  indicated  most  effect- 
ively by  GRE  score. 

Hierarchical  Clustering 
As  was  explained  in  the  study  design,  the  intercorrelation  be- 
tween outcome  expectations  of  the  five  academic  areas  of  the  qualifying 
examination  was  somewhat  low  (Table  2).     This  being  the  case,  greater 
overall  predictive  power  was  achieved  by  developing  a  separate  equa- 
tion for  predicting  each  area  rather  than  developing  one  equation  for 
predicting  their  average.     It  was  also  useful,  however,  to  determine 
the  extent  and  pattern  of  predictive  agreement  in  outcome  measures      ,  . 
between  the  five  areas  on  the  qualifying  examination.     The  technique 

L 

used  for  this  purpose  was  a  hierarchical  clustering  procedure  called 
JAN, 

In  the  JAN  grouping  technique,   the  first  step  was  to  determine 
a  single  squared  multiple  correlation  value  which  indicated  the  overall 
predictive  efficiency  obtained  when  separate  regression  equations  were 
used  for  each  area.     In  the  second  step,  each  equation  was  compared 
with  every  other  equation  to  determine  which  two  could  be  combined 
with  the  minimum  loss  of  predictive  ability.     At  the  completion  of 
step  two,  the  total  number  of  equations  in  the  system  was  reduced 
by  one  and  a  new  overall  squared  multiple  correlation  was  computed. 
The  single  equation  which  replaced  two  of  the  original  equations  best 
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represented,  in  terms  of  the  predictor  variables,  the  joint  outcome 
expectations  of  the  two  academic  areas  clustered.    This  clustering 
procedure  was  continued  in  a  stepwise  manner  until  a  single  predic- 
tion equation  was  generated  to  express  the  combined  outcome  expectations 
of  all  academic  areas  with  as  little  loss  of  predictive  ability  as 
possible. 

An  important  feature  of  this  clustering  technique  was  that  pat- 
terns of  agreement  in  outcome  expectations  of  the  academic  areas  ^ 
could  be  observed.    Examination  of  the  loss  in  overall  predictive 
efficiency  at  the  various  stages  of  the  grouping  process  helped  to 
identify  the  number  and  composition  of  agreement  clusters  between  the 
academic  areas.    Results  of  the  clustering  procedure  applied  to  the 
five  equations  reported  in  the  last  section  are  presented  in  the  fol- 
lowing discussion. 

As  can  be  seen  in  Table  13,  the  overall  predictive  ability  of 
the  five-equation  system  expressed  in  squared  multiple  correlation, 
R^,  was  .22.     This  R^  value  reflected  the  approximate  percentage  of 
prediction  possible  with  the  five-equation  system.    It  is  important 
to  note  that  this  R^  value  indicated  the  average  ability  of  the  system 
to  predict  the  outcome  of  the  qualifying  examination  in  any  individual 
academic  area.     Results  of  each  subsequent  step  of  clustering  were  also 
shown  in  Table  13. 

The  method  for  analyzing  the  groupings  was  to  look  for  breaks 
in  the  overall  R^  value  as  increasingly  dissimilar  areas  were  clustered. 
This  method  was  outlined  by  Ward  &  Hook  (1963).     Applying  this  method 
to  the  data  presented  in  Table  13  seemed  to  indicate  that  optimum 
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prediction  of  the  outcome  measure  in  each  academic  area  in  terms  of 

the  study  variables  required  a  rather  unique  equation.    Though  the 

2 

effects  are  only  slight,  however;  discontinuities  in  the  R  function 
after  the  first  two  groupings  were  somewhat  greater.     This  suggested 
some  possibility  for  jointly  expressing  the  outcome  expectations  of 
three  academic  areas,  educational  administration,  social  foundations 
of  education,  and  curriculum  and  instruction  in  terms  of  the  independ- 
ent variables .    The  two  remaining  academic  areas ,  psychological  founda 
tions  and  personnel  services,  and  historical  and  philosophical  founda- 
tions of  education  each  appeared  to  represent  a  more  clearly  separate 
and  distinct  set  of  outcome  expectations. 

An  important  point  to  note  in  this  clustering  procedure  was  that 
the  order  of  joining  the  cluster  was  independent  of  any  question  of 
validity  of  outcome  expectation.     For  example,  to  assume  that  the 
outcome  expectations  of  the  historical  and  philosophical  foundations 
area  were  in  some  way  less  valid  because  this  area  was  last  to  be 
clustered,     is  unwarranted.     To  illustrate  this  point,  the  outcome 
measure  in  historical  and  philosophical  foundations  was  found  most 
highly  correlated  with  11  of  the  15  indicators  of  academic  success 
shown  in  Table  14. 

If  the  clustering  procedure  was  continued  until  one  single  "best 

equation  for  simulating  all  outcome  expectations  was  produced,  the 
2 

final  R    value  was  .02.     This  value  represented  the  average  ability 

of  the  one  equation  to  predict  the  outcome  expectations  in  all  areas. 

At  the  same  time,  however,   this  equation  predicted  the  average  of 

2 

all  outcome  expectations  with  an  R    of  .37.     To  the  extent  that  a 
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doctoral  student  was  judged  as  passing  or  failing  the  qualifying  exam- 
ination in  the  past  by  the  average  of  all  academic  areas   (see  Appen- 
dix A),  this  single  equation  was  a  useful  one. 

For  expository  purposes,  a  decision  was  made  to  present  only  a 
five-variable  equation  for  the  joint  prediction  of  all  areas.  This 

decision  was  based  on  the  observation  that  beyond  the  fifth  step  of 

2 

regression  increases  in  R    for  each  additional  step  were  relatively 

2 

small  (Table  15).     A  five-step  equation  which  produced  an  R    of  .26 
was  written  from  Appendix  K,  Step  5,  as  follows:  * 

Average  Score  on  the  Qualifying  Exam  = 
+  .4741 

+  .0006  (Verbal  Aptitude  Test  of  GRE) 
+  .0008  (Advanced  Test  of  GRE) 

-  .0647  (Graduation  Rate  Factor  for 

Undergraduate  Degree  Institution) 
+  .7054  (Post -Admission  Graduate  Honor  Point  Average) 

-  .0001  (Square  of  Age) 

Predictive  Qualities  of  Three  Subsets 
As  mentioned  in  the  introduction  to  this  chapter,  the  predictive 
qualities  of  each  of  three  subsets  of  variables  are  discussed  in  this 
section.     First,  a  multiple  prediction  system  generated  from  the  sub- 
set of  pre-admission  variables  is  presented.     Then,  from  the  subset 
of  pos t- admission  variables,  a  second  equation  is  shown.     Next,  a 
third  multiple  regression  equation  derived  from  the  subset  of  program 
variables  is  given.     The  predictive  significance  of  each  of  these 
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subsets  of  variables  in  the  single  "best"  overall  equation  developed 
in  the  last  section  is  also  given. 

Multiple  Prediction  with  Pre-Admission  Variables 

The  53  pre-admission  variables  identified  in  Table  1  and  Appen- 
dix C  were  developed  into  a  separate  multiple  regression  equation  to 

determine  the  predictive  ability  of  this  subset  alone.     From  Table  16 

2 

it  can  be  seen  that  a  maximum  R    of  .27  was  obtained  with  this  group 

2 

of  variables  in  54  steps.    Also  in  Table  16,  an  R    of  .16  was  shown  " 
for  a  five-variable  equation.     The  equation  was  written  from  Appendix 
L,  Step  5,  as  follows: 

Average  Score  on  the  Qualifying  Examination  = 
+1.1456 

-  .00001  (Square  of  Verbal  Aptitude  Test  of  GRE) 
+  .0066     (Verbal  Aptitude  Test  of  GRE) 

+  .0013     (Advanced  Test  of  GRE)  ' 

-  .0730     (Graduation  Rate  Factor  for  Undergraduate 

Degree  Institution) 

-  .0001     (Square  of  Age) 

This  equation  represented  the  maximum  relationship  which  could  be 
obtained  between  the  average  outcome  on  the  qualifying  examination  and 
a  five-variable  combination  from  the  pre-admission  subset.     Using  the 
variance  ratio  test  for  statistical  significance,  the  ability  of  this 
subset  of  variables  alone  to  effectively  predict  the  criterion  was 
found  significant  at  the  .01  level.  . 
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Multiple  Prediction  with  Post -Admission  Variables 

In  Table  1  and  Appendix  C,  ten  post-admission  variables  were 
listed.     When  only  these  variables  were  considered  in  the  multiple 
prediction  of  the  average  outcome  on  the  qualifying  examination,  a. 
maximum  R    of  .19  was  attained  in  seven  steps   (Table  17).     It  can  be 
seen  that  virtually  all  of  the  predictive  ability  was  accounted  for 
in  one  step  of  regression.    A  one-variable  equation  had  an  R    of  .18 
and  was  written  as  below  from  Appendix  M,  Step  1: 

Average  Score  on  the  Qualifying  Examination  =  . 
+  .2660 

+  .8383  (Post-Admission  Graduate  Honor  Point  Average) 

From  the  post-admission  subset  of  variables,  prediction  of  the  average 
outcome  of  all  areas  of  the  qualifying  examination  was  significant 
at  the  .01  level.     The  equation  above  represented  an  optimum  predic- 
tion system  based  on  one  of  the  post-admission  variables. 

Multiple  Prediction  with  Program  Variables  -  ■ 

Multiple  regression  analysis  of  the  15  program  variables  shown  in 

2 

Table  1  and  Appendix  C  resulted  in  a  maximum  R    of  .17  (Table  18). 

Beyond  the  third  step,  however,  each  additional  variable  added  very 

2 

little  to  the  three-predictor  R    value  of  .09.     Appendix  N,  Step  3, 
shows  the  following  equation  for  the  first  three  steps  of  regression 
analysis : 
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Average  Score  on  the  Qualifying  Examination  - 
+1.5241 

+  .2045  (Honor  Point  Average  in  Educational  Adminis- ■,. 

tration  Courses)  ■ 
+  .1833  (Honor  Point  Average  in  Social  Foundations 

Courses) 

+  .1080  (Honor  Point  Average  in  Curriculum  and 
Instruction  Courses) 

The  three-variable  combination  above  allowed  the  maximum  three-variable 
prediction  of  the  average  qualifying  examination  score  from  the  program 
variables.    The  predictive  ability  of  the  subset  of  variables  above 
was  found  significant  at  the  .01  level. 


CHAPTER  V 
DISCUSSION 

It  should  be  remembered  that  the  design  of  the  study  centered 
around  developing  a  multiple  prediction  equation  for  each  of  five 
academic  areas  of  the  doctoral  qualifying  examination  and  then  com- 
bining these  equations  in  a  stepwise  hierarchical  manner.     It  was 
anticipated  that  the  significance  of  the  study  would  be  the  produc- 
tion of  accountability  information  suitable  for  local  policy  making 
and  input  to  national  level  information  systems.    Discussion  of  the 
results  includes:     limitations,  observations,  and  implications  of 
this  study.    A  chapter  summary  is  also  provided. 

Limitations  of  the  Study 
Since  the  primary  statistical  tool  of  the  present  study  was 
regression  analysis,  it  is  very  appropriate  to  review  limiting  factors 
of  this  technique.     One  factor  which  should  be  remembered  is  the  maxi- 
mizing nature  of  the  R  statistic,  Guilford  (1956)  and  Schenck  & 
Nay lor  (1968)  have  cautioned  about  this  characteristic  of  R.    A  second 
factor  to  be  considered  is  the  dependency  of  the  outcome  of  any  pre- 
diction equation  on  the  particular  predictor  variables  investigated. 
A  third  limitation  is  the  extent  to  which  the  study  results  can  be 
generalized  beyond  the  study  population. 
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Although  the  R  values  in  this  study  are  no  doubt  inflated  in 
teriDS  of  prediction,  they  are  quite  useful  for  making  relative  com- 
parisons.   Comparisons  in  R  between  the  various  areas  of  the  qualify- 
ing examination  internal  to  this  study  have  already  been  noted  and  can 
be  viewed  in  Appendix  E.     Another  type  comparison  which  can  be  made 
is  with  other  studies  using  a  statistical  approach  similar  to  ths 
present  study.     The  multiple  correlations  obtained  in  the  present 
study  fell  in  the  same  general  range  as  those  of  similar  studies 
(Dawes,  1971;  Houston  &  Roscoe,  1968;  Platz,  1959;  Robertson  &  Nielsen, 196I; 
Talbott,  1966;  Williams  et  al.,  1969).     In  each  of  these  studies, 
multiple  prediction  of  an  outcome  measure  in  graduate  study  resulted 
in  R  values  ranging  from  .36  to  .64.     The  maximum  R  values  reported 

t 

in  the  results  of  the  present  study  ranged  from  .41  to  .61  and  were 
highly  significant  statistically. 

One  possibility  as  to  why  the  R  values  presented  have  not  com- 
pared even  more  favorably  with  those  of  other  studies  is  that  study 
design  limitations  on  the  types  of  variables  investigated  prevented 
greater  predictive  efficiency.     For  example,  the  absence  of  psycho- 
logical data  such  as  measures  of  interest,  self-concept,  and  motiva- 
tion resulted  in  an  obvious  shortcoming  in  the  study  variables.  Fail- 
ure to  include  such  variables  in  the  design  of  the  study  naturally 
eliminated  them  from  the  prediction  equations. 

As  was  previously  noted,  the  statistical  model  for  this  study 
was  the  fixed-effects  model.     Generalization  beyond  the  study  popula- 
tion is  usually  considered  not  possible  with  this  model  since  the 
independent  variables  need  not  be  normally  distributed.     Christal  (196  8b) 
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suggested  that  a  prediction  equation  can  hold  up  even  if  the  predictors 
are  not  normally  distributed.    For  three  additional  reasons,  the  re- 
sults of  the  present  study  can  be  considered  reasonably  generalizable. 

One  reason  stems  from  the  nature  of  the  study  population.  The 
population  was  relatively  large,  highly  selected,  and  drawn  over  a 
seven-year  period.    A  second  reason  is  the  stability  of  the  graduate 
program,  faculty,  and  institution.     On  the  basis  of  this  reason,  the 
study  results  should  at  least  generalize  to  future  students  in  the 
University  of  Florida  College  of  Education.     The  third  reason,  a  purely 
statistical  one,  is  that  the  number  of  study  cases  exceeded  the  number 
of  predictor  variables  by  a  ratio  of  over  9  to  1. 

Observations  of  the  Study 
In  drawing  together  observations  made  in  this  study,  two  levels 
were  examined.     The  first  level  is  reported  as  observations  about  the 
prediction  equations.     The  second  level  of  observations  reported  deals 
with  some  specific  study  variables. 

Observations  about  the  Prediction  Equations 

The  five  equations  which  were  developed  to  predict  each  of  the 
academic  areas  of  the  qualifying  examination  were  interesting  in  both 
their  similarities  and  their  contrasts.     The  most  obvious  similarity 
among  these  equations  was  the  consistency  and  prominence  of  grades  in 
doctoral  study  as  a  predictor  variable.     The  importance  of  this  vari- 
able was  reflected  in  two  other  equations.     The  overall  equation  for 
prediction  of  the  average  score  on  the  qualifying  examination  was  one 
such  equation.     The  other  equation  was  prediction  of  the  average  score 
from  the  post-admission  subset  of  variables. 


Another  similarity  was  the  occurrence  of  GRE  verbal  aptitude 
score  as  a  variable  in  three  of  the  five  equations.    Again,  this  vari 
able  appeared  in  two  later  equations  predicting  the  average  qualify- 
ing examination  score, 

A  third  similarity  of  note  was  between  the  equations  for  the 
educational  administration  and  curriculum  and  instructions  areas. 
Both  of  these  areas  were  also  separate  departments  in  the  College  of 
Education.    In  each  case,  majoring  in  the  respective  department  was 
a  variable  of  some  importance  in  multiple  prediction.     This  suggested 
that  a  student  received  something  purely  through  contact  with  his 
major  department  that  was  not  reflected  in  number  of  hours  or  grades 
in  departmental  courses. 

Dissimilarities  among  the  five  area  equations  were  obvious 
simply  in  the  fact  that  each  had  a  unique  combination  of  predictor 
variables  and  coefficient  weights.     Though  this  observation  can  be 
made  by  visual  inspection  of  the  various  equations,  the  hierarch- 
ical clustering  procedure  added  confirmation  since  little  predictive 
homogeneity  was  noted  among  the  five  equations. 

As  was  previously  mentioned,   the  overall  equation  for  predic- 
tion of  the  average  score  on  the  qualifying  examination  was  meaningful 
because  one  requirement  for  passing  the  examination  had  been  a  satis- 
factory average  grade  (Appendix  A).     The  equation  presented  provided 
a  five-variable  combination  which  accounted  for  about  26  percent  of  th 
criterion  variance.     The  fact  that  the  average  of  all  academic  areas 
was  more  predictable  from  the  study  variables  than  were  the  individual 
areas  suggested  a  possible  difficulty.     It  is  likely  that  there  was 
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considerable  variation  in  grading  standards  among  the  graders  (perhaps 
also  within  the  graders)  of  each  academic  area.    These  possible  var- 
iations tended  to  be  averaged  out  in  the  joint  prediction  of  all 
areas. 

Prediction  of  the  average  score  on  the  qualifying  examination 
by  each  of  three  subsets  of  variables  was  accomplished  primarily  to 
test  the  predictive  role  of  each  subset.     The  predictive  ability  of 
each  of  these  subsets,  pre-admission  variables,  post-admission  vari- 
ables, and  program  variables,  was  found  significant  at  the  .01  level. 
The  equation  produced  from  each  subset  was  also  interesting,  however. 

The  equation  produced  from  pre-admission  variables  was  a  rela- 
tively convenient  five-step  system  which  predicted  16  percent  of  the 
variation  in  the  average  examination  score.     This  was  exclusively  from 
variables  available  prior  to  admission  for  doctoral  study. 

Predictive  importance  of  the  variable,  grades  in  doctoral  study, 
was  once  more  underscored  by  the  post-admission  variables  equation. 
This  variable  alone,  properly  weighted  and  with  an  appropriate  con- 
stant value,  accounted  for  approximately  18  percent  of  the  variation 
in  the  examination  average.     The  equation  generated  from  three  of  the 
program  variables  subset  had  a  predictive  efficiency  of  only  9  per- 
cent, but  pointed  up  an  interesting  trend.-    In  most  cases,  quality 
of  performance  in  courses  from  various  academic  areas  in  education 
was  a  better  predictor  of  overall  success  than  was  the  number  of 
courses  in  these  areas. 
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Observations  about  the  Study  Variables 

Perhaps  as  important  as  which  variables  were  accepted  into  pre- 
diction equations  was  which  variables  were  not.     Variables  based  on 
letters  of  recommendation,  undergraduate  grades  (particularly  for  the 
last  two  years),  pre-admission  graduate  grades,  quantitative  scores 
on  the  GRE,  and  the  categories  of  holding  a  fellowship,  holding  an 
assistantship,  and  receiving  remedial  study  for  the  qualifying  examin- 
ation rather  surprisingly  showed  little  prominence  in  the  equations. 
Some  clues  as  to  why  were  surmised  by  examining  the  data  in  Tables  4, 

6,  8,  10,  and  12.  ; 

With  the  exception  of  the  quantitative  scores  on  the  GRE,  when 
any  potential  equation  variable  from  these  other  sources  appeared 
among  the  highest  ten  correlates  of  an  area  criterion,  it  also  corre- 
lated rather  highly  with  an  equation  variable.    All  such  potential 
variables,  except  those  based  on  letters  of  recommendation,  appeared 
as  a  high  correlate  at  least  once.     In  almost  every  case  the  equation 
variable  which  absorbed  their  effect  was  grades  in  doctoral  study. 

The  situation  with  the  quantitative  scores  of  the  GRE  was  shown 
fairly  well  in  Table  14.     Not  only  was  it  nonessential  in  multiple 
prediction,  it  showed  only  a  slight  individual  relationship  to  all  area 
criteria.    Apparently,  the  skills  measured  by  the  Quantitative  Aptitude 
Test  of  the  GRE  were  not  related  to  desired  outcomes  in  the  study  popu- 
lation. 

One  type  variable  showed  up  in  a  somewhat  unexpected  way  in  three 
of  the  presented  equations.     The  type  variable  was  the  graduation  rate 
factor  which  was  used  to  classify  undergraduate  institutions  on  the 
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basis  of  the  percentage  of  freshmen  they  graduate.    The  main  reason 
for  including  this  factor  as  a  variable  was  to  generate  possible  inter- 
active effects  between  undergraduate  grades  and  the  type  of  institu- 
tion granting  the  grades.     Instead,  unexpectedly,  the  effect  was  a 
direct  one.    Furthermore,  the  effect  worked  opposite  to  the  intuitive. 
As  was  mentioned  in  the  study  results,  institutional  ratings  reflect- 
ing a  high  attrition  rate  were  negatively  related  to  success  on  the 
criteria.     The  effect  was  not  great,  but  it  was  persistent. 

Possibly  something  like  a  categorical  variable  was  being  felt 
since  the  University  of  Florida  was  in  the  highest  attrition  rating 
level.    Eighty-three  of  the  105  cases  which  received  the  highest 
attrition  rating  for  undergraduate  institution  granting  the  degree 
were  University  of  Florida  graduates.     The  meaning  of  this  factor 
remains  questionable  even  with  this  information.     It  is  tempting  to 
speculate  that  some  psychological  factors  may  have  been  operating. 
A  conservative  approach  suggests  the  possibilities  for  further  in- 
vestigation. 

A  final  consideration  regarding  the  study  variables  is  the 
relevance  of  the  many  generated  variables.    A  total  of  33  variables 
were  generated  by  functions  of  other  variables.     Of  this  number,  only 
two  showed  enough  meaning  in  overall  multiple  prediction  to  take  note 
of.     The  more  prominent  of  the  two  was  the  square  of  age  at  time  of 
admission.     This  squared  value  was  a  better  individual  predictor  than 
was  age  itself.    Apparently,  a  downward  opening  second  order  curve  was 
a  better  representation  than  a  line  of  the  relationship  between  age 
and  performance. 
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The  other  variable  of  the  two  was  the  square  of  verbal  aptitude 
score  on  the  GRE.     Unlike  the  case  with  age,   the  square  of  the  verbal 
aptitude  score  was  not  a  better  single  predictor  than  the  verbal  score 
itself.    However,  in  the  equation  formed  from  pre-admission  variables 
only,  the  square  of  verbal  GRE  score  did  add  slightly  to  multiple  pre- 
diction. 

Based  on  the  rather  poor  showing  of  essentially  all  of  the  gen- 
erated variables,  their  inclusion  in  the  study  seems  pretty  useless 
in  hindsight.     Certainly  the  place  of  such  variables  in  future  studies 
similar  in  nature  to  this  study  should  be  viewed  with  some  skepticism. 

Implications  of  the  Study 

Implications  of  this  study  are  discussed  in  rather  general  terms 
of  accountability.     In  this  respect,  concern  is  directed  toward  identi 
fying  prospects  for  local  policy  considerations.     Additionally,  some 
mention  is  made  of  findings  which  possibly  have  wider  significance. 

The  positive  relationship  between  grades  in  doctoral  study  and 
performance  on  the  qualifying  examination  spoke  well  for  the  consist- 
ency of  faculty  expectations.     Furthermore,  this  relationship  seemed 
to  provide  a  rather  reliable  means  for  diagnosing  difficulty  in  the 
course  of  a  doctoral  program. 

The  uniqueness  of  the  prediction  equations  for  the  five  academic 
areas  suggested  the  utility  of  particularized  admissions  depending  on 
prospective  major.     This  theme  was  reinforced  by  the  difficulty  of 
clustering  the  various  areas  into  common  prediction  systems.  Since 
the  average  performance  on  the  qualifying  examination  was  reasonably 
predictable  by  a  combination  of  five  pre-admission  variables,  some 
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system  such  as  this  seemed  an  efficient  way  to  make  a  minimum  screen- 
ing of  applicants  for  admission.    Additional,    more  particularlized 
admissions  screening,  based  on  prospective  major,  would  seem  very 
profitable. 

Also  in  regard  to  admissions,  the  minimal  relationships  found 
between  quantitative  score  on  the  GRE  and  the  various  criteria  seemed 
noteworthy.    Going  one  step  further,  the  findings  of  this  study  would 
seem  to  raise  a  general  question  as^  to  the  meaning  of  the  quantitative 
score  in  prediction  of  success  in  graduate  study  programs  similar  to 
the  one  investigated. 

As  previously  noted,  the  generated  variables  in  this  study  were 
essentially  nonproductive.    Because  of  the  large  number  of  such  vari- 
ables included,  their  lack  of  productivity  suggested  that  the  procedure 
of  building  variables  like  these  may  not  be  worthwhile. 

A  final  consideration  was  based  on  the  observation  that  the  aver- 
age score  on  the  qualifying  examination  was  more  predictable  than  any 
of  the  individual  academic  areas.    As  mentioned,  this  could  be  a  sign 
that  considerable  variation  existed  in  the  grading  standards  of  any 
one  area.     The  evidence  for  this,  however,  was  not  conclusive.  Ample 
doubt  was  raised  to  warrant  suggesting  the  possibility  of  a  standard- 
ized qualifying  examination  procedure  graded  on  the  basis  of  normed 
data.  .  .  ■  '  '       '  . 

Summary 

In  review,  limitations  of  the  regression  analysis  technique  were 
discussed.     In  spite  of  the  limitations  of  this  technique,  it  was 
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concluded  that  the  study  results  were  suitable  for  internal  compari- 
sons  and  comparisons  with  the  results  of  other  similar  studies.  It 
was  also  concluded  that  the  study  results  could  be  reasonably  gener- 
alized beyond  the  study  population. 

Study  observations  were  made  about  the  prediction  equations  and 
about  the  predictor  variables.    In  regard  to  the  prediction  equations, 
both  similarities  and  contrasts  were  noted  in  comparing  the  five 
area  equations.     The  differences  between  the  prediction  equations  for 
the  five  academic  areas  were  further  demonstrated  by  a  difficulty  in 
clustering  them  into  a  common  prediction  system.    A  practical  five- 
variable  equation  was  produced  as  an  end  result  of  the  hierarchical 
clustering  procedure.     This  equation  accounted  for  26  percent  of  the 
variation  in  the  average  qualifying  examination  score.     It  was  noted 
that  the  average  of  all  areas  on  the  qualifying  examination  was  more 
predictable  than  any  of  the  separate  areas.     The  predictive  qualities 
of  each  of  the  three  subsets  of  variables:     pre-admission  variables, 
post-admission  variables,  and  program  variables,  were  highly  signifi- 

cant  in  overall  prediction. 

In  regard  to  the  predictor  variables,  it  was  observed  that  the 
most  consistent  and  prominent  predictor  variable  was  grades  in  doctoral 
study.     Another  predictor  variable  of  importance  in  multiple  predic- 
tion was  the  verbal  aptitude  score  of  the  GRE.     One  variable  which 
had  a  surprisingly  poor  showing  in  multiple  prediction  as  well  as 
very  little  individual  relationship  to  the  area  criteria  was  quantita- 
tive score  of  the  GRE.     The  33  generated  variables  in  general  also 
showed  poorly  in  multiple  prediction. 
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On  the  basis  of  the  study  observations,  five  implications  were 
cited.     First,  it  was  suggested  that  grades  in  doctoral  study  would 
seem  to  provide  a  very  reliable  means  for  diagnosing  difficulty  in 
the  course  of  a  doctoral  program.    Second,  although  a  general  screen- 
ing process  of  all  applicants  for  admission  could  likely  be  done  quite 
effectively  by  an  equation,  the  unique  prediction  equation  for  each 
area  suggested  that  further  particularized  admissions  based  on  pros- 
pective major  would  seem  profitable.     Third,  the  predictive  meaning 
of  the  quantitative  score  on  the  GRE  in  the  investigated  doctoral  pro- 
gram and  other  similar  programs  was  questioned.    Fourth,  the  utility 
of  generating  combination  variables  in  future  studies  similar  to  this 
study  seemed  doubtful.     Finally,  the  rather  good  liklihood  of  consid- 
erable variation  in  standards  among  the  graders  in  each  academic  area 
of  the  qualifying  examination  indicated  that  a  standardized  testing 
procedure  would  be  worthwhile. 


APPENDICES 


APPENDIX  A 

Procedures  Followed  by  the  Qualifying  Examination 
Committee  for  Administering  Examinations 

1.  The  committee  meets  about  four  to  six  weeks  before  the  examinations 
which  are  given  three  times  a  year  (September,  January,  and  June).  The 
exams  are  started  on  the  Monday  following  the  beginning  of  classes  for 

each  trimester.    The  main  purpose  of  this  meeting  is  to  select  persons  to 
prepare  questions  in  the  following  areas:    Curriculum;  Psychology,  Human 
Development,  and  Personnel  Services;  Sociological  Foundations;  Administra- 
tion; Historical  and  Philosophical  Foundations. 

2.  Two  persons  are  selected  for  each  area  by  the  Qualifying 
Examination  Committee  to  prepare  and  serve  as  readers  for  their  area  of 
specialization.    Whenever  possible,  one  of  the  persons  preparing  the 
questions  should  have  worked  on  the  preceding  examination.    When  this  is 
not  possible,  an  effort  is  made  to  get  at  least  one  veteran  of  earlier 
examinations. 

3.  The  persons  selected  to  write  examination  questions  are 
ordinarily  faculty  members  of  the  College  of  Education  who  have  doctor's 
degrees  and  are  members  of  the  graduate  faculty.    They  are  usually  asked 
to  write  in  their  field  of  specialization. 

4.  After  the  writing  teams  are  selected,  they  are  notified  by  the 
Assistant  Dean  for  graduate  studies  in  education  who  serves  as  an  ex 
officio  member  and  secretary  to  the  Qualifying  Examination  Committee.  A 
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fonn  letter  approved  by  the  Committee  is  sent  to  each  reader  and  appears 
as  Appendix  A  to  this  statement. 

5.  The  names  of  persons  writing  questions  are  kept  confidential, 
both  before  and  after  the  examinations. 

6.  Deadlines  are  named  for  submitting  questions  to  the  Office  of 
the  Assistant  Dean,  College  of  Education.    The  Qualifying  Examination 
Committee  meets  a  second  time  and  reviews  the  questions  for  clarity, 
appropriate  difficulty,  length,  etc.    Minor  modifications  or  editing  may 
be  done  by  the  Committee.    Major  modifications  are  not  made  in  the 
questions  without  the  consent  of  the  writers.    The  writers  may  be  asked 
to  change  the  questions,  or  occasionally,  to  write  new  ones.    Care  is 
taken  to  see  that  there  is  little  overlapping  among  the  questions. 

7.  The  Assistant  Dean  and  the  Chairman  of  the  Qualifying  Examination 
Committee  later  arrange  the  sequence  in  which  the  questions  are  administered. 
Usually,  a  fairly  straightforward  question  is  selected  for  the  first 
question  to  be  administered. 

8.  The  Assistant  Dean  for  graduate  work  in  the  College  of  Education 
is  in  charge  of  administering  the  examination  questions.    Each  student  is 
assigned  a  number.    This  number  alone  appears  on  all  answers  to  the 
questions.    Names  are  revealed  only  after  the  papers  have  been  read  and 
evaluated  and  the  Qualifying  Examination  Committee  has  made  its  decisions 
with  regard  to  a  pass  or  a  failure  in  all  instances. 

9.  The  student's  answers  are  sent  by  confidential  folder  to  the 
writers  of  the  questions  for  evaluation,  along  with  a  rating  sheet  and 
instructions  presented  as  Appendix  B.     After  the  readers  have  read  and 
returned  the  papers,  a  member  of  the  committee  takes  the  rating  sheets 
and  prepares  a  profile  for  each  student.    The  check  marks  are  given 
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numerical  weights--!. 0  for  the  bottom  of  the  scale,  3.0  for  the  midpoint, 
and  5.0  for  the  top.     In  case  of  a  student  who  has  a  profile  ranging  be- 
low 2.70,  a  second  person  is  asked  to  carefully  examine  the  numerical 
weights  assigned.     The  entire  committee  uses  the  rating  sheets  as  returned 
by  the  readers  in  making  its  decision  in  each  case  where  a  student  is  in 
the  borderline  position. 

10.    A  third  meeting  of  the  Qualifying  Examination  Committee  is  held 
for  each  examination  period  to  make  final  judgment  with  regard  to  passing 
and  failing.     The  Committee  has  available  the  rating  sheets,  a  profile  of 
assigned  numerical  weights ,  comments  made  by  the  readers ,  and  the  exami- 
nation papers.     Each  case  is  presented  and  discussed  anonymously.  Ordi- 
narily a  total  numerical  score  below  2.50  or  a  score  less  than  this  in 
more  than  three  areas  is  considered  as  a  failure.    A  score  between  2.50 
and  2.70  is  generally  considered  as  in  a  grey  zone  where  a  committee  judg- 
ment is  made.     The  Committee  has  followed  the  following  procedure  in  most 
examinations : 

A.  A  motion  is  made  and  seconded  and  acted  upon  for  those 
in  the  opinion  of  the  Committee  who  have  clearly  passed  the 
examination. 

B.  A  second  motion  is  acted  upon  for  those  clearly  failing 
the  examination, 

C.  Finally,  the  Comasittee  gives  careful  consideration  and 
acts  individually  upon  each  case  which  falls  between  a  clear  pass 
or  failure.     In  this  consideration  the  Committee  has  available  and 
has  on  numerous  occasions  inspected  the  written  papers  in  addition 
to  a  consideration  of  the  profiles  and  comments  of  the  readers. 

As  noted  above,  no  names  are  revealed  until  the  Committee  makes  its 


final  judgment  for  each  case. 

11.  Students  are  notified  by  special  delivery  letter  from  the  Office 
of  the  Assistant  Dean,  College  of  Education,  immediately  following  the 
committee  decisions.    Information  is  kept  confidential  by  Committee 
members. 

12.  The  papers  for  each  student  are  kept  on  confidential  file  in 
the  Office  of  the  Assistant  Dean.    These  files  are  available  for  inspec- 
tion by  any  interested  faculty  member  on  the  staff  of  the  College  of 
Education.  ^ 
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Student  No._ 
Question  No._ 
Reader 


RATING  SCALE  FOR  THE  DOCTORAL  QUALIFYING  EXAMINATION 

Please  indicate  carefully  on  the  following  scale  your  rating  of  this 
paper.    It  may  be  helpful  to  keep  in  mind  that  the  readers  of  doctoral 
examination  papers  in  the  College  of  Education  have  traditionally  been 
guided  by  the  following  criteria,  among  others: 

I.    Understanding  of  material  and  concepts 

Indicate  familiarity  with  basic  concepts  and  materials  in 
area;  is  precise  in  use  of  concepts  and  ideas,  avoids  use 
of  educational  jargon  and  cliches. 

II.    Clarity  of  Expression 

Keeps  discussion  relevant  to  question;  is  clear  and  concise; 
^       logical  flow  of  ideas. 

III.    Evidence  of  Scholarship 

Knows  basic  sources  and  major  research  in  area;  uses 
references  appropriately;  gives  facts  accurately  and  cited 
generalizations  correctly. 

IV.    Critical  Mindedness 

Supports  beliefs  with  evidence;  evaluates  sources  cited; 
presents  evidence  of  reflection  on  reading. 

V.  Creativity 

Is  sensitive  to  ramifications  pf  problems;  synthesizes  own 
solutions  for  problems;  shows  insight  in  diagnosis;  proposes 
own  solutions. 


1.  2.     2.4  2.5    2.9      3.  4.  5. 

/  I      I  I      I         I  I  I 

ll/ll/lllllllll  mil  lllllllllllllllllllll 
F.                    Q.  P. 
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The  Committee  suggests  that  it  will  be  helpful  if  the  faculty  will  make 
some  comments  regarding  their  appraisal  of  the  paper.    This  is  particularly 
true  where  the  rating  of  the  paper  is  in  the  questionable  area.    Use  the 
space  below  for  this  purpose. 

NOTE:    The  Committee  requests  that  readers  not  write  comments  on  the 
body  of  the  students'  papers. 


REMARKS; 
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APPENDIX  D 

Mean  and  Standard  Deviation  of  Independent  Variables 


Variable 
Number 


Variable 


Mean 


Standard 
Deviation 


1. 
2. 
3. 
4. 
5. 

6. 

7. 

8. 

9. 
10. 
11. 
12. 

13. 


GRE  verbal  score 
GRE  quantitative  score 
GRE  advanced  score 
MAT  score 

Undergraduate  honor  point 
average  for  the  first  two 
years  of  study 

Undergraduate  honor  point 
average  for  the  last  two 
years  of  study 

Undergraduate  major  in 
liberal  arts 

Undergraduate  major  in 
science  or  mathematics 

Undergraduate  major  in 
a  professional  field 

Honor  point  average  in 
Freshman  English 

Number  of  undergraduate 
changes 

Selectivity  factor  for  insti- 
tution where  Freshman  English 
taken 

Selectivity  factor  for  insti- 
tution where  baccalaureate 
degree  received 


573.65 
559.39 
580.83 
60.81 

2.64 

2.94 
.31 
.11 


.59 


2.62 


,78 


5.12 


5.33 


81.22 
95.14 
60.23 
11.72 

.56 

.44 
.46 
.31 
.49 
.77 
1.04 

1.32 

1.03 


-102- 


-103- 


APPENDIX  D  (continued) 


Variable 
Number 


Variable 


Mean 


Standard 
Deviation 


14. 

15. 

16. 
17. 
18. 
19. 
20. 

21. 
22. 

23. 

24. 

25. 
26. 

27. 
28. 

29. 

30. 


Graduation  rate  factor  for 
institution  where  Freshman 
English  taken  5.91 

Graduation  rate  factor  for 
institution  where  baccalau- 
reate degree  received  6.05 

Master's  degree  in  education  .76 

Age  35.15 

Sex  .20 

Prior  military  experience  .46 

High  school  graduate  in 

Florida  .31 

Years  of  teaching  experience  6.14 

Recommendation  ratings  prior 

to  admission  4.04 

Graduate  honor  point  average 

prior  to  admission  3.58 

Master's  degree  received  at 
University  of  Florida  .49 

Time  in  doctoral  study  21.71 

Recommendation  ratings  after 
admission  4.07 

Major  department  Foundations  .14 

Major  department  Special 

Education  .04 

Major  department  Counselor 

Education  .25 

Major  department  Administration  .27 


.94 

.85 
.43 
7.78 
.40 
.50 

.46 

5.84 

.31 

.21 

.50 
18.58 

.28 
.34 

.19 

.43 
.44 
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Variable 
Number  


31. 

32. 
33. 

34. 
35. 
36. 

37. 

38. 
39. 

40. 

41. 

42. 

43. 

44. 

45. 

46. 
47. 


Major  department  Curriculum 
and  Instruction 

Remedial  study 

Graduate  honor  point  average 
after  admission 

Assistantship  held 

Fellowship  held 

Honor  point  average  in 
educational  administration 

Honor  point  average  in 
psychological  foundations 

Honor  point  average  in 
sociological  foundations 

Honor  point  average  in  philo- 
sophical and  historical  foun- 
dations 

Honor  point  average  in 
curriculum  and  instruction 

Total  hours  of  educational 
administration 

Total  hours  of  psychological 
foundations 

Total  hours  of  sociological 
foundations 

Total  hours  of  historical  and 
philosophical  foundations 


.31 
.11 

3.70 
.51 
.40 

3.73 

3.68 

3.70  ' 

3.56 
3.69 
11.03 
18.31 
3.30 
4.02 


Total  hours  of  curriculum 
and  instruction 

Square  of  number  1 

Square  of  number  2 


13.06 

335663.81 
321950.94' 


Standard 
Deviation 


.46 
.32 

.19 
.50 
.49 

.29 

.31 

.43 

.44 
.36 
11.78 
15.69 

3.31 

3.58 

8.54 
93182.88 
108887.00 
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APPENDIX  D  (continued) 


Variable 
Number 

Variable 

Mean 

Standard 
Deviatio 

48. 

Product 

of  numbers  5  and  12 

13.38 

4.32 

49. 

Product 

of  5,  12,  and  14 

78.73 

28.18 

50. 

Product 

of  5  and  14 

15.61 

4.19 

51. 

Product 

of  6  and  13 

15.66 

3.72 

52. 

Product 

of  6,  13,  and  15 

94.47 

25.85 

53. 

Product 

of  6,  13,  15,  and 

7 

28.18 

44.60 

54. 

Product 

of  6,  13,  15,  and 

8 

9.21 

27.82 

55. 

Product 

of  6,  13,  15,  and 

9 

57.13 

51.69 

56. 

Product 

of  6  and  15 

17.81 

3.63 

57. 

Product 

of  6,  15,  and  7 

5.25 

8.21 

58. 

Product 

of  6,  15,  and  8 

1.75 

5.19 

59. 

Product 

of  6,  15,  and  9 

10.83 

9.39 

60. 

Product 

of  6,  13,  and  7 

4.86 

7.62 

61. 

Product 

of  6,  13,  and  8 

1.57 

4.71 

62. 

Product 

of  6,  13,  and  9 

9.24 

8.20 

63. 

Product 

or  0  ana  / 

.90 

1.38 

64. 

Product 

of  6  and  8 

.30 

.87 

65. 

Product 

of  6  and  9 

1.75 

1.50 

66. 

Product 

of  10  and  12 

13.36 

5.27 

67. 

Product 

of  10  and  14 

15.47 

5.32 

68. 

Product 

of  10,  12,  and  14 

78.45 

33.12 

69. 

Square  ( 

3f  17 

1295.91 

603.17 

70. 

Product 

of  23  and  16 

2.73 

1.55 
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APPENDIX  D  (continued) 


Variable 
Number  


71. 

72. 

73. 

74. 

75. 

76. 

77. 

78. 


Variable 


Mean 


Standard 
Deviation 


Product  of  23  and 

1.79 

1.82 

Product  of  23,  16,  and  /a 

1.53 

1.80 

Square  of  25 

815.82 

2300.64 

Product  of  36  and  41 

41.64 

44.83 

Product  of  37  and  42 

67.96 

59.00 

Product  of  38  and  43 

12.14 

12.12 

Product  of  39  and  44 

14.53 

'  13.63 

Product  of  40  and  45 

48.52 

32.21 

APPENDIX  E 


Maximum  Squared  Multiple  Correlation, 
r2,  for  Each  Academic  Area  of  the 
Qualifying  Examination 


 ::  ^  Number  2 

Area  '   2LllS21  ^  — ^ 


Educational  Administration 

Curriculum  and  Introduction 

Historical  and  Philosophical 
Foundations 

Psychological  Foundations 

Social  Foundations 


72  -^^ 

79  .A5 

89  -54 

76  .  .47 

76  .45 
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APPENDIX  F 


Data  for  Multiple  Regression  of 
Educational  Administration 


Stepwise  Coefficient  Values 


Equation 
Parameter 

Step 
1 

Step 
2 

Step 
3 

Step 
4 

Step 
5 

Step 
10 

R 

.2216 

.2823 

.3068 

,3190 

.3297 

.3593 

Constant 

3.3939 

1.4168 

1.0207 

1.2074 

1.2918 

1.3532 

Variable  3 

— 

.  

.0012 

.0012 

.0012 

.0013 

4 

.0036 

11 

-.0476 

16 

-.1032 

20 

-.1056 

-.0929 

30 

.2415 

.2528 

.2607 

.2741 

.2807 

33 

.2908 

.5386 

.4587 

.4317 

.4198 

.3935 

59 

.0041 

68 

-.0018 

69 

-.0001 

-.0001 

-.0001 
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APPENDIX  G 

Data  for  Multiple  Regression  of 
Curriculum  and  Instruction 


 Stepwise  Coefficient  Values  

Equation  Step  Step  Step  Step  Step  Step 

Parameter  1  2  3  4  5  10 — 

R  .2075  .2476  .2623  .2875  .3019  .3489 

Constant  .4618  .4129  .1810  .3111  .8475  -.3400 

Variable  1              —              —  -0008  .0014  .0014  .0078 

4             _-             —  —  -.0084  -.0087  -.0092 

15             -_             —  —  —  -.0779  -.1236 

27             —             —  ~  ~  ~  -.1889 

31              —  .2099  .2090  .2218  .2194  .1963 

33  .7736  .7692  .7111  .7192  .7049  .5990 

35             __             _-  -  -  .      —  .1116 

46             ~             ~  —  —  —  '0000 

60             —             —  —  ~  —  •  -.0081 

67             —             ~  ~  ~  —  .0119 
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APPENDIX  H 


Data  for  Multiple  Regression  of  Historical  and 
Philosophical  Foundations 


Stepwise  Coefficient  Values 


Equation 
Parameter 

Step 
1 

Step 
2 

Step 
3 

Step 
4 

Step 
5 

Step 
10 

R 

.3371 

.3917 

.4070 

.4209 

.4331 

.4674 

Constant 

-1.3002 

-1.8318  - 

1.5253 

-2.5266 

-2,2689 

-2.4369 

Variable 

1 

.0018 

.0017 

.0017 

.0018 

.0013 

.0009 

7 

.4557 

10 

.1094 

23 

.3633 

.3587 

.3384 

28 

-.3843 

-.4028 

33 

1.2481 

1.1151 

1.0864 

1.0094 

.9301 

.8980 

50 

-.0127 

53 

-.0035 

69 

-.0001 

-.0001 

-.0001 

-.0002 
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APPENDIX  I 


Data  for  Multiple  Regression  of 
Psychological  Foundations 


Stepwise  Coefficient  Values 


Equation 
Parameter 

Step 
1 

Step 
2 

Step 
3 

Step 
4 

Step 
5 

Step 
10 

R 

.1843 

.2350 

.2772 

.2950 

.3133 

.3720 

Constant 

.6828 

.1754 

.9935 

1.3735 

1.5728 

-.2164 

V dL  XclU J-C 

.0012 

14 

-.1205 

-.1272 

-.1284 

-.0772 

18 

.2642 

.2157 

.2209 

23 

.5322 

.3308 

33. 

.7342 

.6024 

.5553 

.5147 

.5577 

38 

.2690 

.2875 

.2334 

.1886 

52 

-.0040 

69 

-.0001 

-.0002 

-.0002 

76 

-.0050 
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APPENDIX  J 


Data  for  Multiple  Regression  of 
Social  Foundations 


Stepwise  Coefficient  Values 


Equation 
Parameter 

Step 
1 

Step 
2 

Step 
3 

Step 
4 

Step 
5 

Step 
10 

R 

.2521 

.2774 

.2932 

.3034 

.3129 

.3519 

Constant 

.4341 

.4948 

.2724 

.3253 

.2416 

.2157 

Variable  1 

.  OUU/ 

.  UUUo 

.  uuu  / 

•  UUUO 

11 

-.0458 

-.0461 

-.0523 

21 

-.0076 

23 

.1707 

33 

.7809 

.7051 

.6616 

.6580 

.6993 

.6727 

37 

-.1574 

45 

.0073 

48 

.0164 

.0142 

.0118 

.0113 

.0099 

74 

-.0010 

-.0012 

77 

.0027 
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APPENDIX  K 


Data  for  Multiple  Regression  for  the  Joint 
Prediction  of  All  Areas  of  the 
Qualifying  Examination 


Stepwis 

e  Coefficient  Values 

Equation 
Parameter 

Step 
1 

Step 
2 

Step 
3 

Step 
4 

Step 
5 

Step 
10 

R 

.4201 

.4552 

.4745 

.4948 

.5087  " 

.5409 

Constant 

.2660 

.0147 

.2150 

.6796 

.4741 

-1.0553 

Variable 

1 

.0008 

.0008 

.0008 

.0006 

.0046 

3 

.0008 

.0008 

10 

.0401 

15 

-.0639 

-.0647 

-.0673 

23 

.1457 

33 

.8383 

.7755 

.7567 

.7418 

.7054 

.5802 

38 

.0719 

45 

.0037 

46 

.0000 

69 

-.0001 

-.0001 

-.0001 

-.0001 
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APPENDIX  L 


Data  for  Multiple  Regression  of  the  Subset,  Pre-Admission 
Variables,  in  Prediction  of  All  Areas  of  the 
Qualifying  Examination 


Stepwise  Coefficient  Values 


Ecjuatxon 
Parameter 

DL6p 

1 

2 

O  L 

3 

4 

Sten 
5 

Step 
10 

R 

.2467 

.2953 

.3401 

.3702 

.4035 

.4589 

Constant 

2.6934 

2.2275 

.3138 

.6748 

1.1456 

.0663 

Variable  1 

.0012 

.0008 

.0075 

.0066 

.0066 

.0070 

3 

.0011 

.0012 

.0013 

.0013 

.0011 

5 

.  .0690 

11 

-.0228 

15 

-.0730 

-.0562 

16 

-.0532 

20 

-.0803 

23 

.2584 

46 

-.00001 

-.00001 

-.00001 

-.00001 

69 

-.0001 

-.0001 

-.0001 
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APPENDIX  M 


Data  for  Multiple  Regression  of  the  Subset,  Post-Admission 
Variables,  in  Prediction  of  All  Areas  of  the 
Qualifying  Examination 


Stepwise  Coefficient  Values 


Rfiii^it'  1  on 
Parameter 

Step 

1 

Step 
2 

Step 
3 

Step 
4 

Step 
5 

(Final; 
Step 
7 

R 

.4201 

.4297 

.4354 

.4378 

.4392 

.4402 

Constant 

.2660 

.4688 

.5835 

.6564 

.6213 

.5191 

Variable  26 

.0259 

27 

-.0289 

28 

-.0929 

-.0994 

-.1076 

30 

-.0316 

-.0388 

32 

-.1134 

-.1125 

-.1144 

-.1136 

-.1090 

33 

.8383 

.7869 

.7497 

.7309 

.7422 

.7426 

35 

.0567 

.0578 

.0624 

.0642 
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APPENDIX  N 


Data  for  Multiple  Regression  of  the  Subset,  Program 
Variables,  in  Prediction  of  All  Areas  of  the 
Qualifying  Examination 


Stepwise  Coefficient  Values 


Equation 
Parameter 

Step 
1 

Step 
2 

Step 
3 

Step 
4 

Step 
5 

Step 
10 

R 

.2384 

.2964 

.3121 

.3220 

.  3321 

•  .3648 

Constant 

2.5697 

1.7639 

1.5241 

1.2486 

1.1872 

2.1050 

Variable  36 

.2337 

.2045 

.1994 

.1991 

.1658 

37 

.1007 

.1203 

-.0214 

38 

.2148 

.1968 

.1833 

.1711 

.1761 

.1537 

39 

.0447 

40 

.1080 

.0998 

.0788 

-.0030 

42 

-.0559 

43 

-.0064 

45 

-.0200 

75 

.0147 

78 

.0010 

.0063 
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